Android Surface-GraphicBuffer-BufferQueue
Android的UI控件最终在Surface上进行绘制;Surface要进行绘制,需要申请显存,绘制,提交显存进行显示。
申请显存
Android的显存由两个部分表示,对APP的接口体现为Surface(native/libs/gui/Surface.cpp),对graphics部分(CPU/GPU/OPENGL)体现为GraphicBuffer。
Surface说明
Surface本身有两个含义,一个是代表UI系统的Canvas,另一个是代表本地window系统,为跨平台的OPENGL(EGL)提供接口。
UI一般基于Canvas绘制,参考UI的始祖View的draw函数:
public void draw(Canvas canvas)
所有UI控件继承自View,都会基于Canvs来绘制自己;UI组件的draw是谁触发的,canvas是怎么创建的?这些秘密在ViewRootImpl里面,每个Activity在setContentView之后,系统会为其创建一个ViewRootImpl对象,该对象代替Activity管理其view系统,并和window系统建立关联(Activity的window就是在该类中创建的),并且ViewRootImpl会建立和SurfaceFlinger的连接,监听SurfaceFlinger的VSYNC信号,一旦VSYNC信号发生,ViewRootImpl就会进入到framecallback中进行绘制。其中ViewRootImpl拥有window对应的Surface对象:
private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff, boolean scalingRequired, Rect dirty, Rect surfaceInsets) { // Draw with software renderer. final Canvas canvas;try { ...... canvas = mSurface.lockCanvas(dirty); ...... if (!canvas.isOpaque() || yoff != 0 || xoff != 0) { canvas.drawColor(0, PorterDuff.Mode.CLEAR); } try { canvas.translate(-xoff, -yoff); ....... mView.draw(canvas); } finally { ...... } } finally { try { surface.unlockCanvasAndPost(canvas); } catch (IllegalArgumentException e) ...... }
对于Canvas的使用流程:
Surface.lockCanvas->View.draw(Canvas)-> Surface.unlockCanvasAndPost(Canvas)
在Surface.lockCanvas中会调用native的对象android_view_Surface.cpp->Surface.dequeueBuffer->BufferQueueProducer.dequeueBuffer得到struct ANativeWindowBuffer 的对象,其实就是一个GraphicBuffer对象,与此同时还返回了FenceID。
ANativeWindow_Buffer outBuffer; status_t err = surface->lock(&outBuffer, dirtyRectPtr); if (err < 0) { const char* const exception = (err == NO_MEMORY) ? OutOfResourcesException : "java/lang/IllegalArgumentException"; jniThrowException(env, exception, NULL); return 0; } SkImageInfo info = SkImageInfo::Make(outBuffer.width, outBuffer.height, convertPixelFormat(outBuffer.format), outBuffer.format == PIXEL_FORMAT_RGBX_8888 ? kOpaque_SkAlphaType : kPremul_SkAlphaType, GraphicsJNI::defaultColorSpace()); SkBitmap bitmap; ssize_t bpr = outBuffer.stride * bytesPerPixel(outBuffer.format); bitmap.setInfo(info, bpr); if (outBuffer.width > 0 && outBuffer.height > 0) { bitmap.setPixels(outBuffer.bits); } else { // be safe with an empty bitmap. bitmap.setPixels(NULL); } Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj); nativeCanvas->setBitmap(bitmap);
由上面的lockCanvas代码片段来看,根据ANativeWindowBuffer构建了一个SKBitmap对象,将该对象设置给nativeCanvas(SkiaCanvas),然后就返回到Java空间了。
上面提到了Canvas,我们看一下Canvas的处理流程。
大部分基于Canvas的操作最后会落到SKCanvas上面去,这个在Skia 2D库里面。如果想搞清楚流程,可以拿TextView或者Android任意一个UI控件,看一下他的draw是怎么利用canvas API来实现的。也可以看一下skia库实现。
提交显存
在UI绘制完成后,需要将绘制的内容提交显示,这里用到了Surface::unlockAndPost:
status_t Surface::unlockAndPost() { if (mLockedBuffer == 0) { ALOGE("Surface::unlockAndPost failed, no locked buffer"); return INVALID_OPERATION; } int fd = -1; status_t err = mLockedBuffer->unlockAsync(&fd); ALOGE_IF(err, "failed unlocking buffer (%p)", mLockedBuffer->handle); err = queueBuffer(mLockedBuffer.get(), fd); ALOGE_IF(err, "queueBuffer (handle=%p) failed (%s)", mLockedBuffer->handle, strerror(-err)); mPostedBuffer = mLockedBuffer; mLockedBuffer = 0; return err; }
其中主要就是将GraphicBuffer 提交到BufferQueue上等待SurfaceFlinger(comsumer)显示出来。
GraphicBufferProducer诞生流程
BufferQueue的基本结构如下:
GraphicBuffer就与基于BufferQueueProducer产生的,在Surface.cpp里面有一个sp<IGraphicsBufferProducer> mGraphicBufferProducer;所有对GraphicBuffer的queue/dequeue/cancel等都是通过mBufferProducer产生的,我们看一下这个对象是怎么产生,谁在server端为其服务,client和server的连接是怎么建立的。
从前面Surface的说明里面我们提到一点,就是ViewRootImpl;ViewRootImpl里面的Surface为所有View的绘制提供canvas,我们看一下这个Surface是怎么创建的就能搞清楚Surface.mGraphicBufferProducer是怎么实例化的。ViewRootImpl是Activity View管理者,也是Activity对应window的创建者,在其中有几个步骤:
create window,就是创建Activity对应的window对象,是和WMS建立通讯创建窗口
try { mOrigWindowType = mWindowAttributes.type; mAttachInfo.mRecomputeGlobalAttributes = true; collectViewAttributes(); res = mWindowSession.addToDisplay(mWindow, mSeq, mWindowAttributes, getHostVisibility(), mDisplay.getDisplayId(), mWinFrame, mAttachInfo.mContentInsets, mAttachInfo.mStableInsets, mAttachInfo.mOutsets, mAttachInfo.mDisplayCutout, mInputChannel); } catch (RemoteException e) { mAdded = false;
relayout window,测量窗口大小位置等:
int relayoutResult = mWindowSession.relayout(mWindow, mSeq, params, (int) (mView.getMeasuredWidth() * appScale + 0.5f), (int) (mView.getMeasuredHeight() * appScale + 0.5f), viewVisibility, insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0, frameNumber, mWinFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets, mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingDisplayCutout, mPendingMergedConfiguration, mSurface);
在WMS里面上面两个接口分别调用addWindow以及relayoutWindow;其中relayoutWindow中创建了实际的surface,也就是说实在WMS中显示窗口的时候去创建了实际的surface,其创建过程如下:
最后是调用surface.copyFrom(SurfaceControl)得到真实的surface;SurfaceControl是在WMS里面创建的,SurfaceControl创建的时候就会向SurfaceComposerClient申请创建surface:
static jlong nativeCreate(JNIEnv* env, jclass clazz, jobject sessionObj, jstring nameStr, jint w, jint h, jint format, jint flags, jlong parentObject, jint windowType, jint ownerUid) { ScopedUtfChars name(env, nameStr); sp<SurfaceComposerClient> client(android_view_SurfaceSession_getClient(env, sessionObj)); SurfaceControl *parent = reinterpret_cast<SurfaceControl*>(parentObject); sp<SurfaceControl> surface; status_t err = client->createSurfaceChecked( String8(name.c_str()), w, h, format, &surface, flags, parent, windowType, ownerUid); if (err == NAME_NOT_FOUND) { jniThrowException(env, "java/lang/IllegalArgumentException", NULL); return 0; } else if (err != NO_ERROR) { jniThrowException(env, OutOfResourcesException, NULL); return 0; } surface->incStrong((void *)nativeCreate); return reinterpret_cast<jlong>(surface.get()); }
在createSurfaceChecked里面想surfaceFlinger申请创建Surface,并基于创建的Surface创建新的SurfaceControl。然后一步步返回,ViewRootImpl里面的Surface就具备真正的显存了。但是我们前面是要知道GraphicBufferProducer是谁创建的,这个秘密就在ComposerSurfaceClient.createSurfaceChecked函数里面。ComposerSurfaceClient有一个成员变量mClient,这是SurfaceFlinger.Client的客户端,通过这个mClient和SurfaceFlinger建立通讯。
status_t SurfaceComposerClient::createSurfaceChecked(...) { sp<SurfaceControl> sur; status_t err = mStatus; if (mStatus == NO_ERROR) { sp<IBinder> handle; sp<IBinder> parentHandle; sp<IGraphicBufferProducer> gbp; if (parent != nullptr) { parentHandle = parent->getHandle(); } err = mClient->createSurface(name, w, h, format, flags, parentHandle, windowType, ownerUid, &handle, &gbp); ALOGE_IF(err, "SurfaceComposerClient::createSurface error %s", strerror(-err)); if (err == NO_ERROR) { *outSurface = new SurfaceControl(this, handle, gbp, true /* owned */); } } return err; }
在这里可以看到调用了mClient的createSurface,然后返回了gbp(也就是IGraphicBufferProducer);mClient是SurfaceFlinger.Client的客户端,由此可见GraphicBufferProducer实际是有SurfaceFlinger进程创建的。mClient和SurfaceFlinger的对象关系如下图所示:
那么进入SurfaceFlinger看一下到底是怎么创建GraphicBufferProducer的;Client.createSurface->SurfaceFlinger.createLayer-> new BufferLayer,实际是在BufferLayer::onFirstRef里面创建的:
void BufferLayer::onFirstRef() { // Creates a custom BufferQueue for SurfaceFlingerConsumer to use sp<IGraphicBufferProducer> producer; sp<IGraphicBufferConsumer> consumer; BufferQueue::createBufferQueue(&producer, &consumer, true); mProducer = new MonitoredProducer(producer, mFlinger, this); mConsumer = new BufferLayerConsumer(consumer, mFlinger->getRenderEngine(), mTextureName, this); mConsumer->setConsumerUsageBits(getEffectiveUsage(0)); mConsumer->setContentsChangedListener(this); mConsumer->setName(mName); if (mFlinger->isLayerTripleBufferingDisabled()) { mProducer->setMaxDequeuedBufferCount(2); } const sp<const DisplayDevice> hw(mFlinger->getDefaultDisplayDevice()); updateTransformHint(hw); }
由上面的创建过程也可以看出来,Surface提交的GraphicBuffer由BufferLayerConsumer来消耗。
posted on 2019-08-29 16:20 wanglifangfang 阅读(1953) 评论(0) 编辑 收藏 举报