Hi! Here is MelonTe.|

MelonTe

园龄:4个月粉丝:3关注:0

Golang的Channel机制源码学习

1、核心数据结构

1.1、hchan

type hchan struct {
	qcount   uint           // total data in the queue
	dataqsiz uint           // size of the circular queue
	buf      unsafe.Pointer // points to an array of dataqsiz elements
	elemsize uint16
	closed   uint32
	elemtype *_type // element type
	sendx    uint   // send index
	recvx    uint   // receive index
	recvq    waitq  // list of recv waiters
	sendq    waitq  // list of send waiters

	// lock protects all fields in hchan, as well as several
	// fields in sudogs blocked on this channel.
	//
	// Do not change another G's status while holding this lock
	// (in particular, do not ready a G), as this can deadlock
	// with stack shrinking.
	lock mutex
}
  • qcount:当前channel已存储的的元素数量
  • dataqsiz:channel的环形队列(缓冲区)的大小
  • buf:指向缓冲区的指针
  • elemsize:元素的大小
  • closed:表示channel是否关闭
  • elemtype:元素的类型
  • sendx:下一个发送操作的元素在队列中的位置
  • recvx:下一个接受操作的位置
  • recvq:等待从channel接收数据的goroutine队列
  • sendq:等待向channel发送数据的goroutine队列
  • lock:锁

1.2、waitq

type waitq struct {
	first *sudog
	last  *sudog
}

waitq是阻塞的G队列,有一个指向头部和一个指向尾部的指针。

1.3、sudog

type sudog struct {
	g *g

	next *sudog
	prev *sudog
	elem unsafe.Pointer // data element (may point to stack)

	//...

	// isSelect indicates g is participating in a select, so
	// g.selectDone must be CAS'd to win the wake-up race.
	isSelect bool
	//...
	c        *hchan // channel
}

sudog(pseudo-G)(伪G)用于包装一个G,表示在等待队列中的一个G。需要sudog是因为G和同步对象之间的关系是多对多的,一个G可以出现在多个等待队列中,一个等待队列也可以有多个G,因此需要sudog建立G和Chan之间的联系。

  • isSelect:标识当前协程是否处于select多路复用的流程中

2、构造器

func makechan(t *chantype, size int) *hchan {
	elem := t.Elem

	//...
    //判断内存申请是否合理
	mem, overflow := math.MulUintptr(elem.Size_, uintptr(size))
	if overflow || mem > maxAlloc-hchanSize || size < 0 {
		panic(plainError("makechan: size out of range"))
	}

	var c *hchan
	switch {
        //无缓冲channel
	case mem == 0:
		// Queue or element size is zero.
		c = (*hchan)(mallocgc(hchanSize, nil, true))
		// Race detector uses this location for synchronization.
		c.buf = c.raceaddr()
        //无指针缓冲类型channel
	case elem.PtrBytes == 0:
		// Elements do not contain pointers.
		// Allocate hchan and buf in one call.
		c = (*hchan)(mallocgc(hchanSize+mem, nil, true))
		c.buf = add(unsafe.Pointer(c), hchanSize)
	default:
		// Elements contain pointers.
		c = new(hchan)
		c.buf = mallocgc(mem, elem, true)
	}

	c.elemsize = uint16(elem.Size_)
	c.elemtype = elem
	c.dataqsiz = uint(size)
	lockInit(&c.lock, lockRankHchan)

	if debugChan {
		print("makechan: chan=", c, "; elemsize=", elem.Size_, "; dataqsiz=", size, "\n")
	}
	return c
}

makechan首先会判断要申请开辟的空间是否合法,接着会根据要缓冲的元素类型和大小分为三种chan

  • 无缓冲的channel
  • 有缓冲,但是不包含指针元素的channel
  • 有缓冲,且包含指针元素的channel

为什么要分类是否包含指针元素呢?

主要是为了垃圾回收内存优化,对于不包含指针元素的chan,可以对hchan+buf进行一次性的分配,它们的内存空间是连续的,避免多次mallocgc调用,也可以跳过GC的扫描。而包含指针元素的chan,就需要单独分配buf,因为GC需要追踪指针。

3、写流程

3.1、异常处理

func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
	if c == nil {
		if !block {
			return false
		}
		gopark(nil, nil, waitReasonChanSendNilChan, traceBlockForever, 2)
		throw("unreachable")
	}

	//...

	if c.closed != 0 {
		unlock(&c.lock)
		panic(plainError("send on closed channel"))
	}
    //...
  • 向未初始化的chan发送数据,会导致死锁
  • 对于已经关闭的chan发送数据,会导致panic

3.2、case1:写时存在堵塞的读协程

func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
    
lock(&c.lock)
//...
if sg := c.recvq.dequeue(); sg != nil {
		// Found a waiting receiver. We pass the value we want to send
		// directly to the receiver, bypassing the channel buffer (if any).
		send(c, sg, ep, func() { unlock(&c.lock) }, 3)
		return true
	}

如果存在阻塞的读协程,会直接将元素拷贝给对应的goroutine。

3.3、case2:写时无阻塞读协程并且缓冲区有空间

func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
    
lock(&c.lock)
    
    if c.qcount < c.dataqsiz {
		// Space is available in the channel buffer. Enqueue the element to send.
		qp := chanbuf(c, c.sendx)
		typedmemmove(c.elemtype, qp, ep)
		c.sendx++
		if c.sendx == c.dataqsiz {
			c.sendx = 0
		}
		c.qcount++
		unlock(&c.lock)
		return true
	}

将当前的元素添加到环形缓冲区的sendx的位置上。

3.3、case3:写时无阻塞读协程并且缓冲区无空间

func chansend(c *hchan, ep unsafe.Pointer, block bool, callerpc uintptr) bool {
    
lock(&c.lock)
    // Block on the channel. Some receiver will complete our operation for us.
	gp := getg()
	mysg := acquireSudog()
	// No stack splits between assigning elem and enqueuing mysg
	// on gp.waiting where copystack can find it.
	mysg.elem = ep
	mysg.waitlink = nil
	mysg.g = gp
	mysg.isSelect = false
	mysg.c = c
	gp.waiting = mysg
	gp.param = nil
	c.sendq.enqueue(mysg)

	gp.parkingOnChan.Store(true)
	gopark(chanparkcommit, unsafe.Pointer(&c.lock), waitReasonChanSend, traceBlockChanSend, 2)
	// Ensure the value being sent is kept alive until the
	// receiver copies it out. The sudog has a pointer to the
	// stack object, but sudogs aren't considered as roots of the
	// stack tracer.
	KeepAlive(ep)

	// someone woke us up.
	if mysg != gp.waiting {
		throw("G waiting list is corrupted")
	}
	gp.waiting = nil
	gp.activeStackChans = false
	closed := !mysg.success
	gp.param = nil
	if mysg.releasetime > 0 {
		blockevent(mysg.releasetime-t0, 2)
	}
	mysg.c = nil
	releaseSudog(mysg)
	if closed {
		if c.closed == 0 {
			throw("chansend: spurious wakeup")
		}
		panic(plainError("send on closed channel"))
	}
	return true
}

我们需要阻塞当前的协程,用于等待机会来向通道发送消息。

  • mysg是制造的一个sudog对象,包装了当前的G,然后建立G和Chan之间的关系。
  • c.sendq.enqueue(mysg)sudog添加到当前channel的阻塞写协程队列;
  • 将当前的协程阻塞,调用了gopark。
  • 倘若协程被唤醒,则回收掉sudog

4、读流程

4.1、异常处理

func chanrecv(c *hchan, ep unsafe.Pointer, block bool) (selected, received bool) {
	// raceenabled: don't need to check ep, as it is always on the stack
	// or is new memory allocated by reflect.

	if c == nil {
		if !block {
			return
		}
		gopark(nil, nil, waitReasonChanReceiveNilChan, traceBlockForever, 2)
		throw("unreachable")
	}

	if !block && empty(c) {
		
		if atomic.Load(&c.closed) == 0 {
			return
		}
		if empty(c) {
			// The channel is irreversibly closed and empty.
			if ep != nil {
				typedmemclr(c.elemtype, ep)
			}
			return true, false
		}
	}
    if c.closed != 0 {
		if c.qcount == 0 {
			if raceenabled {
				raceacquire(c.raceaddr())
			}
			unlock(&c.lock)
			if ep != nil {
				typedmemclr(c.elemtype, ep)
			}
			return true, false
		}
		// The channel has been closed, but the channel's buffer have data.
  • 管道为空,会导致死锁
  • 读已经关闭的管道会直接返回

4.2、case1:读时存在阻塞的写协程

lock(&c.lock)

if sg := c.sendq.dequeue(); sg != nil {
			// Found a waiting sender. If buffer is size 0, receive value
			// directly from sender. Otherwise, receive from head of queue
			// and add sender's value to the tail of the queue (both map to
			// the same buffer slot because the queue is full).
			recv(c, sg, ep, func() { unlock(&c.lock) }, 3)
			return true, true
		}
  • 存在阻塞的写协程,则:
  • 若缓冲区大小为0,则直接获取写协程的元素并且唤醒写协程
  • 否则,读取缓冲队列头部的元素,然后唤醒写协程将消息写到缓冲区尾部
  • 解锁

4.3、case2:读时不存在阻塞的写协程,并且缓冲区有元素

if c.qcount > 0 {
		// Receive directly from queue
		qp := chanbuf(c, c.recvx)
		if ep != nil {
			typedmemmove(c.elemtype, ep, qp)
		}
		typedmemclr(c.elemtype, qp)
		c.recvx++
		if c.recvx == c.dataqsiz {
			c.recvx = 0
		}
		c.qcount--
		unlock(&c.lock)
		return true, true
	}
  • 直接获取缓冲区的元素
  • 读取计数器+1
  • 若刚好读完一个环的大小,则置零

4.4、case3:读时不存在阻塞的写协程,并且缓冲区无元素

gp := getg()
	mysg := acquireSudog()
	mysg.releasetime = 0
	if t0 != 0 {
		mysg.releasetime = -1
	}
	// No stack splits between assigning elem and enqueuing mysg
	// on gp.waiting where copystack can find it.
	mysg.elem = ep
	mysg.waitlink = nil
	gp.waiting = mysg
	mysg.g = gp
	mysg.isSelect = false
	mysg.c = c
	gp.param = nil
	c.recvq.enqueue(mysg)
	// Signal to anyone trying to shrink our stack that we're about
	// to park on a channel. The window between when this G's status
	// changes and when we set gp.activeStackChans is not safe for
	// stack shrinking.
	gp.parkingOnChan.Store(true)
	gopark(chanparkcommit, unsafe.Pointer(&c.lock), waitReasonChanReceive, traceBlockChanRecv, 2)

	// someone woke us up
	if mysg != gp.waiting {
		throw("G waiting list is corrupted")
	}
	gp.waiting = nil
	gp.activeStackChans = false
	if mysg.releasetime > 0 {
		blockevent(mysg.releasetime-t0, 2)
	}
	success := mysg.success
	gp.param = nil
	mysg.c = nil
	releaseSudog(mysg)
	return true, success
}

和写流程的case3类似,创建一个sudog,绑定当前的changgopark进行阻塞,等待被唤醒。当唤醒时需要回收sudog

5、阻塞与非阻塞

在上述探索的情况,都是阻塞型channel的情况。而对于非阻塞的channel,会根据函数传入的block参数进行划分,若block为false,则所有使得协程会进入阻塞、造成死锁的流程都会提前返回false。所有能立即完成读取/写入操作的条件下,非阻塞模式下会返回 true。

那么何时进入非阻塞模式呢?

默认情况下,chan都是阻塞模式,只有进行select多路复用分支中,才会变成非阻塞模式。

	ch := make(chan int)
select{
  case <- ch:
  default:
}

底层代码:

func selectnbsend(c *hchan, elem unsafe.Pointer) (selected bool) {
    return chansend(c, elem, false, getcallerpc())
}

func selectnbrecv(elem unsafe.Pointer, c *hchan) (selected, received bool) {
    return chanrecv(c, elem, false)
}

6、关闭通道

func closechan(c *hchan) {
	if c == nil {
		panic(plainError("close of nil channel"))
	}

	lock(&c.lock)
	if c.closed != 0 {
		unlock(&c.lock)
		panic(plainError("close of closed channel"))
	}

	if raceenabled {
		callerpc := getcallerpc()
		racewritepc(c.raceaddr(), callerpc, abi.FuncPCABIInternal(closechan))
		racerelease(c.raceaddr())
	}

	c.closed = 1

	var glist gList

	// release all readers
	for {
		sg := c.recvq.dequeue()
		if sg == nil {
			break
		}
		if sg.elem != nil {
			typedmemclr(c.elemtype, sg.elem)
			sg.elem = nil
		}
		if sg.releasetime != 0 {
			sg.releasetime = cputicks()
		}
		gp := sg.g
		gp.param = unsafe.Pointer(sg)
		sg.success = false
		if raceenabled {
			raceacquireg(gp, c.raceaddr())
		}
		glist.push(gp)
	}

	// release all writers (they will panic)
	for {
		sg := c.sendq.dequeue()
		if sg == nil {
			break
		}
		sg.elem = nil
		if sg.releasetime != 0 {
			sg.releasetime = cputicks()
		}
		gp := sg.g
		gp.param = unsafe.Pointer(sg)
		sg.success = false
		if raceenabled {
			raceacquireg(gp, c.raceaddr())
		}
		glist.push(gp)
	}
	unlock(&c.lock)

	// Ready all Gs now that we've dropped the channel lock.
	for !glist.empty() {
		gp := glist.pop()
		gp.schedlink = 0
		goready(gp, 3)
	}
}
  • 关闭空管道会引起panic
  • 关闭已经关闭的管道会引起panic
  • 将所有阻塞在当前管道的读、写goroutine都添加到glist中
  • 唤醒glist当中的所有协程

7、参阅:

跟着大佬一起学:

Golang Channel 实现原理

本文作者:MelonTe

本文链接:https://www.cnblogs.com/MelonTe/p/18703202

版权声明:本作品采用知识共享署名-非商业性使用-禁止演绎 2.5 中国大陆许可协议进行许可。

posted @   MelonTe  阅读(10)  评论(0编辑  收藏  举报
点击右上角即可分享
微信分享提示
评论
收藏
关注
推荐
深色
回顶
收起