UE 像素流 点击模型 发送消息从虚幻引擎到前端播放器页面
在前面:
https://www.cnblogs.com/makalochen/p/17803468.html#新建游戏模式
我们已经实现了,从前端到UE 的通信,那能不能UE 主动给前端 发送消息呢?
就比如我在播放的像素流中点击了某个 物体 给所有页面发个消息,说你已经点击了某个物体
答案是可以的,参考官网
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/
前置工作
启动信令服务器
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/PixelStreamingIntro/
其实就是个node 服务,启动就完事了
前端
声明一下 ,这部分前端是从信令服务器抠出来的,方便前后端分离
主要就是这些东西
myUI.html
<!DOCTYPE html>
<html>
<head>
<!-- 该文件处理浏览器和虚幻引擎应用间的通信,接受并显示来自服务器的媒体流。在非必要的情况下,请勿修改此JavaScript文件 -->
<script type="text/javascript" src="../scripts/webRtcPlayer.js"></script>
<!-- 此文件将设置处理键盘、鼠标和触摸事件的事件监听器 -->
<script type="text/javascript" src="../scripts/app.js"></script>
<link rel="shortcut icon" href="../images/favicon.ico" type="image/x-icon">
<link rel="icon" type="../image/png" sizes="96x96" href="/images/favicon-96x96.png">
<link rel="icon" type="../image/png" sizes="32x32" href="/images/favicon-32x32.png">
<link rel="icon" type="../image/png" sizes="16x16" href="/images/favicon-16x16.png">
<link type="text/css" rel="stylesheet" href="player.css">
<title>myUI</title>
</head>
<body onload="load()">
<div>
</div>
<div id="playerUI">
<div id="player"></div>
<div id="overlay" class="overlay text-light bg-dark">
<div>
<div id="qualityStatus" class="greyStatus">●</div>
<div id="overlayButton">+</div>
</div>
<div id="overlaySettings">
<div id="kickOthers">
<div class="settings-text">Kick all other players</div>
<label class="btn-overlay">
<input type="button" id="kick-other-players-button" class="overlay-button btn-flat" value="Kick">
</label>
</div>
<div id="fillWindow">
<div class="settings-text">Enlarge Display to Fill Window</div>
<label class="tgl-switch">
<input type="checkbox" id="enlarge-display-to-fill-window-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
</div>
<div id="qualityControlOwnership">
<div class="settings-text">Quality control ownership</div>
<label class="tgl-switch">
<input type="checkbox" id="quality-control-ownership-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<br>
<!-- 编码设置 -->
<section id="encoderSettings">
<div class="settings-text">Encoder Settings</div>
<div id="encoderParamsContainer" class="collapse">
<div class="form-group">
<label for="encoder-rate-control" class="settings-text">Rate Control</label>
<select id="encoder-rate-control">
<option value="CBR" selected>CBR</option>
<option value="VBR">VBR</option>
<option value="ConstQP">ConstQP</option>
</select>
<label for="encoder-target-bitrate-text">Target Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-target-bitrate-text" value="0" min="0" max="100000" />
<label for="encoder-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-max-bitrate-text" value="0" min="0" max="100000" />
<label for="encoder-min-qp-text">Min QP</label>
<input type="number" class="form-control" id="encoder-min-qp-text" value="0" min="0" max="999" />
<label for="encoder-max-qp-text">Max QP</label>
<input type="number" class="form-control" id="encoder-max-qp-text" value="0" min="0" max="999" />
<label for="encoder-multipass" class="settings-text">Multipass</label>
<select id="encoder-multipass">
<option value="DISABLED" selected>DISABLED</option>
<option value="QUARTER">QUARTER</option>
<option value="FULL">FULL</option>
</select>
<div class="settings-text">Filler Data</div>
<label class="tgl-switch">
<input type="checkbox" id="encoder-filler-data-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<input id="encoder-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
</div>
<br>
</section>
<!-- webRTC 设置 -->
<section id="webRTCSettings">
<div class="settings-text">WebRTC Settings</div>
<div id="webrtcParamsContainer" class="collapse">
<div class="form-group">
<label for="webrtc-degradation-pref">Degradation Pref</label>
<select id="webrtc-degradation-pref">
<option value="BALANCED">BALANCED</option>
<option value="MAINTAIN_FRAMERATE">MAINTAIN_FRAMERATE</option>
<option value="MAINTAIN_RESOLUTION">MAINTAIN_RESOLUTION</option>
</select>
<label for="webrtc-max-fps-text">Max FPS</label>
<input type="number" class="form-control" id="webrtc-max-fps-text" value="1" min="1" max="999" />
<label for="webrtc-min-bitrate-text">Min Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-min-bitrate-text" value="0" min="0" max="100000" />
<label for="webrtc-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-max-bitrate-text" value="0" min="0" max="100000" />
<label for="webrtc-low-qp-text">Low QP Threshold</label>
<input type="number" class="form-control" id="webrtc-low-qp-text" value="0" min="0" max="999" />
<label for="webrtc-high-qp-text">High QP Threshold</label>
<input type="number" class="form-control" id="webrtc-high-qp-text" value="0" min="0" max="999" />
</div>
<input id="webrtc-params-submit" class="btn btn-primary btn-lg mt-3" type="button" value="Apply">
</div>
</section>
<br>
<!-- 显示FPS -->
<div id="showFPS">
<div class="settings-text">Show FPS</div>
<label class="btn-overlay">
<input type="button" id="show-fps-button" class="overlay-button btn-flat" value="Toggle">
</label>
</div>
<div id="matchViewportResolution">
<div class="settings-text">Match Viewport Resolution</div>
<label class="tgl-switch">
<input type="checkbox" id="match-viewport-res-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<div id="statsPanel">
<div class="settings-text">Show Stats</div>
<label class="tgl-switch">
<input type="checkbox" id="show-stats-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
<div id="statsContainer" class="statsContainer">
<div id="stats" class="stats"></div>
</div>
<br>
</div>
<div id="latencyTest">
<div class="settings-text">Latency Stats</div>
<label class="btn-overlay">
<input type="button" id="test-latency-button" class="overlay-button btn-flat" value="Test Latency">
</label>
<div id="latencyStatsContainer" class="statsContainer">
<div id=LatencyStats class="stats">No stats yet...</div>
</div>
</div>
</div>
</div>
</div>
</body>
</script>
</html>
player.css
/*Copyright Epic Games, Inc. All Rights Reserved.*/
:root {
/*Using colour scheme https://color.adobe.com/TD-Colors---Option-3-color-theme-10394433/*/
--colour1:#2B3A42;
--colour2:#3F5765;
--colour3:#BDD4DE;
--colour4:#EFEFEF;
--colour5:#FF5035;
--buttonFont:Helvetica;
--inputFont:Helvetica;
}
body{
margin: 0px;
background-color: black;
}
#playerUI {
width: 100%;
position: absolute;
/*top: 0;
left: 0;*/
z-index: 10;
}
.statsContainer {
background-color: rgba(0,0,0,0.8);
text-align: left;
display: block;
margin-top: 5px;
}
.stats {
font-size: 14px;
font-weight: bold;
padding: 6px;
color: lime;
}
canvas{
image-rendering: crisp-edges;
position: absolute;
}
video{
position: absolute;
width: 100%;
height: 100%;
}
#player{
width: 1280px;
height: 720px;
position: relative;
background-color: #000;
}
#overlay{
-moz-border-radius-bottomright: 5px;
-moz-border-radius-bottomleft: 5px;
-webkit-border-bottom-right-radius: 5px;
-webkit-border-bottom-left-radius: 5px;
border-bottom-right-radius: 5px; /* future proofing */
border-bottom-left-radius: 5px; /* future proofing */
-khtml-border-bottom-right-radius: 5px; /* for old Konqueror browsers */
-khtml-border-bottom-left-radius: 5px; /* for old Konqueror browsers */
-webkit-touch-callout: none; /* iOS Safari */
-webkit-user-select: none; /* Safari */
-khtml-user-select: none; /* Konqueror HTML */
-moz-user-select: none; /* Firefox */
-ms-user-select: none; /* Internet Explorer/Edge */
user-select: none; /* Non-prefixed version, currently
supported by Chrome and Opera */
position: absolute;
padding: 4px;
top: 0;
right: 2%;
z-index: 100;
border: 2px solid var(--colour4);
border-top-width: 0px;
}
.overlay {
background-color: var(--colour2);
font-family: var(--buttonFont);
font-weight: lighter;
color: var(--colour4);
}
.overlay-shown > #overlaySettings {
padding: 50px 4px 4px 4px;
display: block;
}
.overlay-shown > div > #overlayButton {
transform: rotate(-135deg);
-webkit-transform: rotate(-135deg); /* Safari */
-moz-transform: rotate(-135deg); /* Firefox */
-ms-transform: rotate(-135deg); /* IE */
-o-transform: rotate(-135deg); /* Opera */
}
#overlayButton:hover{
cursor: pointer;
}
#overlayButton{
transition-duration: 250ms;
float: right;
text-align: right;
font-size: 40px;
}
#qualityStatus{
float: left;
font-size: 37px;
padding-right: 4px;
}
#overlaySettings{
width: 400px;
display: none;
}
.greyStatus {
color: grey;
}
.limeStatus {
color: lime;
}
.orangeStatus {
color: orange;
}
.redStatus {
color: red;
}
#videoMessageOverlay{
z-index: 20;
color: var(--colour4);;
font-size: 1.8em;
position: absolute;
margin: auto;
font-family: var(--inputFont);;
width: 100%;
}
#videoPlayOverlay{
z-index: 30;
position: absolute;
color: var(--colour4);
font-size: 1.8em;
font-family: var(--inputFont);
width: 100%;
height: 100%;
background-color: rgba(100, 100, 100, 0.7);
}
/* State for element to be clickable */
.clickableState{
align-items: center;
justify-content: center;
display: flex;
cursor: pointer;
}
/* State for element to show text, this is for informational use*/
.textDisplayState{
display: flex;
}
/* State to hide overlay, WebRTC communication is in progress and or is playing */
.hiddenState{
display: none;
}
#playButton{
display: inline-block;
height: auto;
}
img#playButton{
max-width: 241px;
width: 10%;
}
#UIInteraction{
position: fixed;
}
#UIInteractionButtonBoundary{
padding: 2px;
}
#UIInteractionButton{
cursor: pointer;
}
#hiddenInput{
position: absolute;
left: -10%; /* Although invisible, push off-screen to prevent user interaction. */
width: 0px;
opacity: 0;
}
#editTextButton{
position: absolute;
height: 40px;
width: 40px;
}
.settings-text{
color: var(--colour4);
vertical-align: middle;
font-size: 18px;
font-weight: normal;
display: inline-block;
}
.overlay-button{
line-height: 1.1;
padding: 1px 6px;
}
.btn-overlay{
float: right;
vertical-align: middle;
display: inline-block;
}
.btn-flat{
background: var(--colour4);
border: 2px solid var(--colour5);
font-weight: bold;
cursor: pointer;
font-family: var(--buttonFont);
font-size: 10px;
color: var(--colour5);
border-radius: 5px;
height: 20px;
}
.btn-flat:disabled{
background: var(--colour4);
border-color: var(--colour3);
color: var(--colour3);
cursor: default;
}
.btn-flat:active{
border-color: var(--colour2);
color: var(--colour2);
}
.btn-flat:focus{
outline: none;
}
/*** Toggle Switch styles ***/
.tgl-switch {
float: right;
vertical-align: middle;
display: inline-block;
}
.tgl-switch .tgl {
display:none;
}
.tgl, .tgl:after, .tgl:before, .tgl *, .tgl *:after, .tgl *:before, .tgl + .tgl-slider {
-webkit-box-sizing: border-box;
box-sizing: border-box;
}
.tgl::-moz-selection, .tgl:after::-moz-selection, .tgl:before::-moz-selection, .tgl *::-moz-selection, .tgl *:after::-moz-selection, .tgl *:before::-moz-selection, .tgl + .tgl-slider::-moz-selection {
background: none;
}
.tgl::selection, .tgl:after::selection, .tgl:before::selection, .tgl *::selection, .tgl *:after::selection, .tgl *:before::selection, .tgl + .tgl-slider::selection {
background: none;
}
.tgl-slider {
float:right;
}
.tgl + .tgl-slider {
outline: 0;
display: block;
width: 40px;
height: 18px;
position: relative;
cursor: pointer;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}
.tgl + .tgl-slider:after, .tgl + .tgl-slider:before {
position: relative;
display: block;
content: "";
width: 50%;
height: 100%;
}
.tgl + .tgl-slider:after {
left: 0;
}
.tgl + .tgl-slider:before {
display: none;
}
.tgl-flat + .tgl-slider {
padding: 2px;
-webkit-transition: all .2s ease;
transition: all .2s ease;
background: #fff;
border: 3px solid var(--colour4);
border-radius: 2em;
}
.tgl-flat + .tgl-slider:after {
-webkit-transition: all .2s ease;
transition: all .2s ease;
background: var(--colour4);
content: "";
border-radius: 1em;
}
.tgl-flat:checked + .tgl-slider {
border: 3px solid var(--colour5);
}
.tgl-flat:checked + .tgl-slider:after {
left: 50%;
background: var(--colour5);
}
/*** Toggle Switch styles ***/
#encoderSettings, #webRTCSettings {
margin: 10px 0px;
}
#encoderParamsContainer, #webrtcParamsContainer {
padding-left: 5%;
}
#encoder-params-submit {
float: right;
}
#webrtc-params-submit {
float: right;
}
select {
text-align: right;
}
input {
text-align: right;
}
.form-group {
display: grid;
grid-template-columns: 50% 50%;
}
.form-group label {
color: var(--colour4);
vertical-align: middle;
font-size: 18px;
font-weight: normal;
}
#latencyTest {
display: block;
}
#latencyTest button {
margin: 30px 0px;
}
#freezeFrameOverlay {
background-color: transparent;
}
.freezeframeBackground {
background-color: #000 !important;
}
除了这两个文件 ,还有一个文件
app.js 修改像素流流送地址
修改成对应的地址即可
ue 创建基础项目 并 显示鼠标
随便创建个第三人称游戏项目,名称就叫 myUEToHtml
创建完成
但是这个鼠标运行之后,鼠标没有显示出来,可以参考这个现实鼠标
https://www.cnblogs.com/makalochen/p/17700166.html
到现在 前置工作已完成
ue 给静态网格体添加鼠标点击事件
现在在ue 中,随便添加个 静态网格体,给它添加一个鼠标点击事件
添加静态网格体
添加点击事件
选中盒体,添加点击事件
然后再关卡蓝图中就可以看到事件
随便打印一句话,运行看看
没反应,将鼠标悬浮在这个事件上面,提示
启用点击事件
在玩家控制器中启用了点击事件后,鼠标悬停此actor并点击左键将进行调用
那我们在玩家控制器中启用点击事件
运行一下看看
模型OnClick事件不生效的两种原因
参考:
https://blog.csdn.net/qqQQqsadfj/article/details/126083284
1.玩家控制器中点击事件没启用
2.模型的碰撞没设置好
ue 向 html 发送消息
参考
https://docs.unrealengine.com/4.27/zh-CN/SharingAndReleasing/PixelStreaming/CustomPlayer/
发送之后回调函数不调的,可能的原因
如上图,是从玩家控制器里去拿PixelStreamerInput 组件,那必须在玩家控制器中,添加组件
myUI.html
<!DOCTYPE html>
<html>
<head>
<!-- 该文件处理浏览器和虚幻引擎应用间的通信,接受并显示来自服务器的媒体流。在非必要的情况下,请勿修改此JavaScript文件 -->
<script type="text/javascript" src="../scripts/webRtcPlayer.js"></script>
<!-- 此文件将设置处理键盘、鼠标和触摸事件的事件监听器 -->
<script type="text/javascript" src="../scripts/app.js"></script>
<link rel="shortcut icon" href="../images/favicon.ico" type="image/x-icon">
<link rel="icon" type="../image/png" sizes="96x96" href="/images/favicon-96x96.png">
<link rel="icon" type="../image/png" sizes="32x32" href="/images/favicon-32x32.png">
<link rel="icon" type="../image/png" sizes="16x16" href="/images/favicon-16x16.png">
<link type="text/css" rel="stylesheet" href="player.css">
<title>myUI</title>
</head>
<body onload="load()">
<div>
<button onclick="test()">测试</button>
<button onclick="reg()">注册</button>
</div>
<div id="playerUI">
<div id="player"></div>
<div id="overlay" class="overlay text-light bg-dark">
<div>
<div id="qualityStatus" class="greyStatus">●</div>
<div id="overlayButton">+</div>
</div>
<div id="overlaySettings">
<div id="kickOthers">
<div class="settings-text">Kick all other players</div>
<label class="btn-overlay">
<input type="button" id="kick-other-players-button" class="overlay-button btn-flat"
value="Kick">
</label>
</div>
<div id="fillWindow">
<div class="settings-text">Enlarge Display to Fill Window</div>
<label class="tgl-switch">
<input type="checkbox" id="enlarge-display-to-fill-window-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
</div>
<div id="qualityControlOwnership">
<div class="settings-text">Quality control ownership</div>
<label class="tgl-switch">
<input type="checkbox" id="quality-control-ownership-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<br>
<!-- 编码设置 -->
<section id="encoderSettings">
<div class="settings-text">Encoder Settings</div>
<div id="encoderParamsContainer" class="collapse">
<div class="form-group">
<label for="encoder-rate-control" class="settings-text">Rate Control</label>
<select id="encoder-rate-control">
<option value="CBR" selected>CBR</option>
<option value="VBR">VBR</option>
<option value="ConstQP">ConstQP</option>
</select>
<label for="encoder-target-bitrate-text">Target Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-target-bitrate-text" value="0" min="0"
max="100000" />
<label for="encoder-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="encoder-max-bitrate-text" value="0" min="0"
max="100000" />
<label for="encoder-min-qp-text">Min QP</label>
<input type="number" class="form-control" id="encoder-min-qp-text" value="0" min="0"
max="999" />
<label for="encoder-max-qp-text">Max QP</label>
<input type="number" class="form-control" id="encoder-max-qp-text" value="0" min="0"
max="999" />
<label for="encoder-multipass" class="settings-text">Multipass</label>
<select id="encoder-multipass">
<option value="DISABLED" selected>DISABLED</option>
<option value="QUARTER">QUARTER</option>
<option value="FULL">FULL</option>
</select>
<div class="settings-text">Filler Data</div>
<label class="tgl-switch">
<input type="checkbox" id="encoder-filler-data-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<input id="encoder-params-submit" class="btn btn-primary btn-lg mt-3" type="button"
value="Apply">
</div>
<br>
</section>
<!-- webRTC 设置 -->
<section id="webRTCSettings">
<div class="settings-text">WebRTC Settings</div>
<div id="webrtcParamsContainer" class="collapse">
<div class="form-group">
<label for="webrtc-degradation-pref">Degradation Pref</label>
<select id="webrtc-degradation-pref">
<option value="BALANCED">BALANCED</option>
<option value="MAINTAIN_FRAMERATE">MAINTAIN_FRAMERATE</option>
<option value="MAINTAIN_RESOLUTION">MAINTAIN_RESOLUTION</option>
</select>
<label for="webrtc-max-fps-text">Max FPS</label>
<input type="number" class="form-control" id="webrtc-max-fps-text" value="1" min="1"
max="999" />
<label for="webrtc-min-bitrate-text">Min Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-min-bitrate-text" value="0" min="0"
max="100000" />
<label for="webrtc-max-bitrate-text">Max Bitrate (kbps)</label>
<input type="number" class="form-control" id="webrtc-max-bitrate-text" value="0" min="0"
max="100000" />
<label for="webrtc-low-qp-text">Low QP Threshold</label>
<input type="number" class="form-control" id="webrtc-low-qp-text" value="0" min="0"
max="999" />
<label for="webrtc-high-qp-text">High QP Threshold</label>
<input type="number" class="form-control" id="webrtc-high-qp-text" value="0" min="0"
max="999" />
</div>
<input id="webrtc-params-submit" class="btn btn-primary btn-lg mt-3" type="button"
value="Apply">
</div>
</section>
<br>
<!-- 显示FPS -->
<div id="showFPS">
<div class="settings-text">Show FPS</div>
<label class="btn-overlay">
<input type="button" id="show-fps-button" class="overlay-button btn-flat" value="Toggle">
</label>
</div>
<div id="matchViewportResolution">
<div class="settings-text">Match Viewport Resolution</div>
<label class="tgl-switch">
<input type="checkbox" id="match-viewport-res-tgl" class="tgl tgl-flat">
<div class="tgl-slider"></div>
</label>
</div>
<div id="statsPanel">
<div class="settings-text">Show Stats</div>
<label class="tgl-switch">
<input type="checkbox" id="show-stats-tgl" class="tgl tgl-flat" checked>
<div class="tgl-slider"></div>
</label>
<div id="statsContainer" class="statsContainer">
<div id="stats" class="stats"></div>
</div>
<br>
</div>
<div id="latencyTest">
<div class="settings-text">Latency Stats</div>
<label class="btn-overlay">
<input type="button" id="test-latency-button" class="overlay-button btn-flat"
value="Test Latency">
</label>
<div id="latencyStatsContainer" class="statsContainer">
<div id=LatencyStats class="stats">No stats yet...</div>
</div>
</div>
</div>
</div>
</div>
</body>
<script>
function test(){
// 发送
emitUIInteraction(`测试发送`);
// 注册监听器函数
addResponseEventListener("handle_responses", (data) => {
// ue 返回
console.log(`myHandleResponseFunction ----------> ${data}`);
});
}
function reg(){
// 注册监听器函数
addResponseEventListener("handle_responses", (data) => {
// ue 返回
console.log(`myHandleResponseFunction2 ----------> ${data}`);
});
}
</script>
</html>
ue 关卡蓝图
上图最主要的 获取玩家控制器 -> 按类获取组件(前提是你在玩家控制器蓝图中,已经将像素流组件添加进去了)