WebRTC实时通信系列教程5 RTCPeerConnection传输视频

【转载请注明出处: http://blog.csdn.net/leytton/article/details/76691543】

 PS：如果本文对您有帮助，请点个赞让我知道哦~微笑

《WebRTC实时通信系列教程》翻译自《Real time communication with WebRTC》

示例代码下载http://download.csdn.net/detail/leytton/9923708

WebRTC实时通信系列教程1 介绍

WebRTC实时通信系列教程2 概述

WebRTC实时通信系列教程3 获取示例代码

WebRTC实时通信系列教程4 从摄像头获取视频流

WebRTC实时通信系列教程5 RTCPeerConnection传输视频

WebRTC实时通信系列教程6 使用RTCDataChannel传输数据

WebRTC实时通信系列教程7 使用Socket.IO搭建信令服务器交换信息

WebRTC实时通信系列教程8 打通P2P连接和信令通信

WebRTC实时通信系列教程9 数据通道图片传输

WebRTC实时通信系列教程10 恭喜完成本系列课程


一、译文

1、大纲

在这一节中你将弄明白如何:

  • 利用 adapter.js 解决WebRTC在浏览器中的兼容问题.
  • 使用RTCPeerConnection API传输视频.
  • 控制媒体捕获和传输.

这一节的示例代码在 step-2 文件夹中.

2、什么是RTCPeerConnection?

RTCPeerConnection是WebRTC调用进行音视频传输和数据通信的API.

这个案例将在同一页面中的两个RTCPeerConnection对象(peers)之间建立连接。

没有什么实际用途,但有利于理解RTCPeerConnection是如何工作的。

3、添加video标签和控制按钮

在 index.html 文件里，放置两个video标签和三个按钮:

<video id="localVideo" autoplay></video>
<video id="remoteVideo" autoplay></video>

<div>
  <button id="startButton">Start</button>
  <button id="callButton">Call</button>
  <button id="hangupButton">Hang Up</button>
</div>

一个video标签将会用于展示 getUserMedia() 获取到的视频流另一个video标签将会用于展示 RTCPeerconnection 获取到的视频流. (也就是一个用于展示本地视频流一个用于展示远程视频流)

4、引入adapter.js文件

在 main.js 文件引入前引入 adapter.js 文件:

<script src="js/lib/adapter.js"></script>

Index.html 现在是这个样子:

<!DOCTYPE html>
<html>

<head>

  <title>Realtime communication with WebRTC</title>

  <link rel="stylesheet" href="css/main.css" />

</head>

<body>

  <h1>Realtime communication with WebRTC</h1>

  <video id="localVideo" autoplay></video>
  <video id="remoteVideo" autoplay></video>

  <div>
    <button id="startButton">Start</button>
    <button id="callButton">Call</button>
    <button id="hangupButton">Hang Up</button>
  </div>

  <script src="js/lib/adapter.js"></script>
  <script src="js/main.js"></script>

</body>

</html>

5、修改RTCPeerConnection代码

把 main.js 替换成 step-02 文件夹里的版本.

并不建议直接copy, 但为了跑通RTCPeerConnection坚持到底那也是没办法的事啦.

等会我们将会解释代码原理.

6、创建连接

打开 index.html 文件, 点击【Start】 按钮来获取你的本地摄像头, 再点击 【Call】 按钮来创建点对点连接.你应该能在另一个video标签看到与你摄像头相同的视频. 查看浏览器控制台可以看到WebRTC日志.

7、工作原理

这里就说来话长了...

WebRTC 在其客户端之间使用 RTCPeerConnection API 来创建视频流连接, 称作 peers.

在这个案例中, 两个 RTCPeerConnection 对象为 pc1 和 pc2. 

在peers之间创建通话需要三个步骤:

  • 在peers两端分别创建 RTCPeerConnection , 并通过getUserMedia()添加本地视频流.
  • 获取并分享网络信息: 这些待连接的终端被称为 ICE 候选者(candidates).
  • 获取并分享本地和远程描述(descriptions): 关于本地媒体的 SDP 格式元数据.

你可以想像 Alice 和 Bob 想要通过 RTCPeerConnection 进行视频聊天.

首先, Alice 和 Bob 交换网络信息.  '寻找候选者'(finding candidates) 正如通过 ICE 框架寻找网络地址和端口的过程.

  1. Alice 创建一个 RTCPeerConnection 对象，这个对象有个 onicecandidate 事件. 这对应了 main.js 文件中的如下代码:
pc1 = new RTCPeerConnection(servers);
trace('Created local peer connection object pc1');
pc1.onicecandidate = function(e) {
  onIceCandidate(pc1, e);
};

这个 servers 参数 在本例中没有用到.你可以用它来指定STUN和TURN服务器

WebRTC 被设计为点对点工作模式, 所以用户之间是尽可能地通过最短路线进行连接. 然而, 在现实世界当中: 客户端应用需要穿透 NAT 网关 和防火墙, 并且点对点网络需要握手来防止直接连接失败.

在这一过程中, WebRTC APIs 使用STUN服务器来获取计算机IP地址和TURN服务器来保证点对点连接成功.详情请看 WebRTC in the real world .

  1. Alice 调用 getUserMedia() 并通过以下语句添加视频流:
pc1.addStream(localStream);
  1. 当网络候选(network candidates)可用时，步骤1的 onicecandidate 将会被调用.
  2. Alice 发送序列化的候选数据(candidate data)给 Bob. 在真实应用中, 这个过程 (称作信令传输 signaling) 通过消息服务器(websokect等)进行 – 在后面的小节中我们将演示如何做. 当然在这个实验小节中, 两个 RTCPeerConnection 对象在同一页面可以直接通信不需要额外的消息服务器.
  3. 当 Bob 从 Alice 那获取到候选数据, 他会调用 addIceCandidate(), 来添加候选者(candidate) 到远程 peer description:
function onIceCandidate(pc, event) {
  if (event.candidate) {
    getOtherPc(pc).addIceCandidate(
      new RTCIceCandidate(event.candidate)
    ).then(
      function() {
        onAddIceCandidateSuccess(pc);
      },
      function(err) {
        onAddIceCandidateError(pc, err);
      }
    );
    trace(getName(pc) + ' ICE candidate: \n' + event.candidate.candidate);
  }
}

WebRTC peers 同样需要获取并交换本地和远程音视频媒体信息, 例如分辨率和编码器功能. 信令通信通过交换媒体blob元数据配置信息, 称作一次 offer 和一次 answer, 通信使用会话描述协议(Session Description Protocol) 格式, 即 SDP.

  1. Alice 调用 RTCPeerConnection createOffer() 方法. JS的promise对象会返回一个 RTCSessionDescription: Alice的本地会话描述（local session description）:
pc1.createOffer(
    offerOptions
  ).then(
    onCreateOfferSuccess,
    onCreateSessionDescriptionError
  );
  1. 如果成功, Alice 调用 setLocalDescription() 设置本地会话描述并且通过信令通道发送这个会话描述给Bob.
  2. Bob 调用 setRemoteDescription() 来设置Alice发送给他的会话描述.
  3. Bob 调用 RTCPeerConnection 的 createAnswer() 方法, 传递从Alice获取的远程描述, 那么能够产生回应Alice的本地会话.  createAnswer() 方法中的promise对象返回一个RTCSessionDescription: Bob 将其设为本地描述并发送给Alice.
  4. 当Alice获取到Bob的会话描述, 她调用setRemoteDescription()将其设置为远程描述. 

function onCreateOfferSuccess(desc) {
  pc1.setLocalDescription(desc).then(
    function() {
      onSetLocalSuccess(pc1);
    },
    onSetSessionDescriptionError
  );
  pc2.setRemoteDescription(desc).then(
    function() {
      onSetRemoteSuccess(pc2);
    },
    onSetSessionDescriptionError
  );
  // Since the 'remote' side has no media stream we need
  // to pass in the right constraints in order for it to
  // accept the incoming offer of audio and video.
  pc2.createAnswer().then(
    onCreateAnswerSuccess,
    onCreateSessionDescriptionError
  );
}

function onCreateAnswerSuccess(desc) {
  pc2.setLocalDescription(desc).then(
    function() {
      onSetLocalSuccess(pc2);
    },
    onSetSessionDescriptionError
  );
  pc1.setRemoteDescription(desc).then(
    function() {
      onSetRemoteSuccess(pc1);
    },
    onSetSessionDescriptionError
  );
}
  1. 搞定!

PS:这里补充一张原理图，摘自《WebRTC手记之初探》


8、扩展

  1. chrome浏览器打开 chrome://webrtc-internals. 它提供了WebRTC统计和调试数据所有的 Chrome URLs. (打开chrome://about能列出所有的Chrome内部地址)
  2. 页面CSS样式:
  • 让视频并列.
  • 让按钮有相同宽度和更大字体.
  • 确保移动端显示兼容性.
  1. 通过Chrome开发工具控制台, 查看 localStream, pc1 和 pc2.
  2. 通过控制台, 查看 pc1.localDescription. 观察SDP格式

9、你学到的

在这一节中你学习了如何:

  • 利用 adapter.js 解决WebRTC在浏览器中的兼容问题.
  • 使用RTCPeerConnection API传输视频.
  • 控制媒体捕获和传输.
  • 客户端之间分享媒体和网络信息来创建WebRTC通信

这一节的示例代码在 step-2 文件夹中.

10、拓展

  • 这一节要学习的东西太多了! 你可以在 webrtc.org/start 找到更多介绍RTCPeerConnection的资源, 这个页面有一些Javascript框架的建议 — 如果你想要使用 WebRTC, 而不想研究它的接口.
  • 你可以在adapter.js GitHub repo 找到更多关于adapter.js的知识.
  • 想要知道世界上最好的视频聊天应用什么样子? 请看 AppRTC, 它是WebRTC项目的典型应用: app, code. 通信建立时间小于500 ms.

11、最佳实践

  • 使用新的基于Promise的APIs， adapter.js 解决浏览器兼容性.

12、接下来

这一节展示了使用WebRTC在peers之间进行视频传输

在下一节中将展示如何使用RTCDataChannel传输任意数据。


二、原文

摘自https://codelabs.developers.google.com/codelabs/webrtc-web/#4


5. Stream video with RTCPeerConnection

What you'll learn

In this step you'll find out how to:

  • Abstract away browser differences with the WebRTC shim, adapter.js.
  • Use the RTCPeerConnection API to stream video.
  • Control media capture and streaming.

A complete version of this step is in the step-2 folder.

What is RTCPeerConnection?

RTCPeerConnection is an API for making WebRTC calls to stream video and audio, and exchange data.

This example sets up a connection between two RTCPeerConnection objects (known as peers) on the same page.

Not much practical use, but good for understanding how RTCPeerConnection works.

Add video elements and control buttons

In index.html replace the single video element with two video elements and three buttons:

<video id="localVideo" autoplay></video>
<video id="remoteVideo" autoplay></video>

<div>
  <button id="startButton">Start</button>
  <button id="callButton">Call</button>
  <button id="hangupButton">Hang Up</button>
</div>

One video element will display the stream from getUserMedia()and the other will show the same video streamed via RTCPeerconnection. (In a real world application, one video element would display the local stream and the other the remote stream.)

Add the adapter.js shim

Add a link to adapter.js above the link to main.js:

<script src="js/lib/adapter.js"></script>

adapter.js is a shim to insulate apps from spec changes and prefix differences.

In fact, the standards and protocols used for WebRTC implementations are highly stable, and there are only a few prefixed names.

In this step, we've linked to a local copy of the most recent version of adapter.js — fine for a codelab, but not good practice for a production app. The adapter.js GitHub repo explains techniques for making sure your app always accesses the most recent version.

For full WebRTC interop information, see webrtc.org/web-apis/interop.

Index.html should now look like this:

<!DOCTYPE html>
<html>

<head>

  <title>Realtime communication with WebRTC</title>

  <link rel="stylesheet" href="css/main.css" />

</head>

<body>

  <h1>Realtime communication with WebRTC</h1>

  <video id="localVideo" autoplay></video>
  <video id="remoteVideo" autoplay></video>

  <div>
    <button id="startButton">Start</button>
    <button id="callButton">Call</button>
    <button id="hangupButton">Hang Up</button>
  </div>

  <script src="js/lib/adapter.js"></script>
  <script src="js/main.js"></script>

</body>

</html>

Install the RTCPeerConnection code

Replace main.js with the version in the step-02 folder.

It's not ideal doing cut-and-paste with large chunks of code in a codelab, but in order to get RTCPeerConnection up and running, there's no alternative but to go the whole hog.

We'll explain how the code works in a moment.

Make the call

Open index.html, click the Start button to get video from your webcam, and click Call to make the peer connection. You should see the same video (from your webcam) in both video elements. View the browser console to see WebRTC logging.

How it works

This step does a lot...

If you want to skip the explanation below, that's fine.

You can still continue with the codelab!

WebRTC uses the RTCPeerConnection API to set up a connection to stream video between WebRTC clients, known aspeers.

In this example, the two RTCPeerConnection objects are on the same page: pc1 and pc2. Not much practical use, but good for demonstrating how the APIs work.

Setting up a call between WebRTC peers involves three tasks:

  • Create a RTCPeerConnection for each end of the call and, at each end, add the local stream fromgetUserMedia().
  • Get and share network information: potential connection endpoints are known as ICE candidates.
  • Get and share local and remote descriptions: metadata about local media in SDP format.

Imagine that Alice and Bob want to use RTCPeerConnection to set up a video chat.

First up, Alice and Bob exchange network information. The expression 'finding candidates' refers to the process of finding network interfaces and ports using the ICE framework.

  1. Alice creates an RTCPeerConnection object with an onicecandidate handler. This corresponds to the following code from main.js:
pc1 = new RTCPeerConnection(servers);
trace('Created local peer connection object pc1');
pc1.onicecandidate = function(e) {
  onIceCandidate(pc1, e);
};

The servers argument to RTCPeerConnection isn't used in this example.

This is where you could specify STUN and TURN servers.

WebRTC is designed to work peer-to-peer, so users can connect by the most direct route possible. However, WebRTC is built to cope with real-world networking: client applications need to traverse NAT gateways and firewalls, and peer to peer networking needs fallbacks in case direct connection fails.

As part of this process, the WebRTC APIs use STUN servers to get the IP address of your computer, and TURN servers to function as relay servers in case peer-to-peer communication fails. WebRTC in the real world explains in more detail.

  1. Alice calls getUserMedia() and adds the stream passed to that:
pc1.addStream(localStream);
  1. The onicecandidate handler from step 1. is called when network candidates become available.
  2. Alice sends serialized candidate data to Bob. In a real application, this process (known as signaling) takes place via a messaging service – we'll show how to do that in a later step. Of course, in this step, the two RTCPeerConnection objects are on the same page and can communicate directly with no need for external messaging.
  3. When Bob gets a candidate message from Alice, he calls addIceCandidate(), to add the candidate to the remote peer description:
function onIceCandidate(pc, event) {
  if (event.candidate) {
    getOtherPc(pc).addIceCandidate(
      new RTCIceCandidate(event.candidate)
    ).then(
      function() {
        onAddIceCandidateSuccess(pc);
      },
      function(err) {
        onAddIceCandidateError(pc, err);
      }
    );
    trace(getName(pc) + ' ICE candidate: \n' + event.candidate.candidate);
  }
}

WebRTC peers also need to find out and exchange local and remote audio and video media information, such as resolution and codec capabilities. Signaling to exchange media configuration information proceeds by exchanging blobs of metadata, known as an offer and an answer, using the Session Description Protocol format, known as SDP:

  1. Alice runs the RTCPeerConnection createOffer() method. The promise returned provides an RTCSessionDescription: Alice's local session description:
pc1.createOffer(
    offerOptions
  ).then(
    onCreateOfferSuccess,
    onCreateSessionDescriptionError
  );
  1. If successful, Alice sets the local description using setLocalDescription() and then sends this session description to Bob via their signaling channel.
  2. Bob sets the description Alice sent him as the remote description using setRemoteDescription().
  3. Bob runs the RTCPeerConnection createAnswer() method, passing it the remote description he got from Alice, so a local session can be generated that is compatible with hers. The createAnswer() promise passes on an RTCSessionDescription: Bob sets that as the local description and sends it to Alice.
  4. When Alice gets Bob's session description, she sets that as the remote description withsetRemoteDescription(). 

function onCreateOfferSuccess(desc) {
  pc1.setLocalDescription(desc).then(
    function() {
      onSetLocalSuccess(pc1);
    },
    onSetSessionDescriptionError
  );
  pc2.setRemoteDescription(desc).then(
    function() {
      onSetRemoteSuccess(pc2);
    },
    onSetSessionDescriptionError
  );
  // Since the 'remote' side has no media stream we need
  // to pass in the right constraints in order for it to
  // accept the incoming offer of audio and video.
  pc2.createAnswer().then(
    onCreateAnswerSuccess,
    onCreateSessionDescriptionError
  );
}

function onCreateAnswerSuccess(desc) {
  pc2.setLocalDescription(desc).then(
    function() {
      onSetLocalSuccess(pc2);
    },
    onSetSessionDescriptionError
  );
  pc1.setRemoteDescription(desc).then(
    function() {
      onSetRemoteSuccess(<span class=%
posted @ 2017-08-05 10:47  Leytton  阅读(1451)  评论(0)  编辑  收藏  举报