海康PS码流分析
背景：通过使用海康SDK和EHOME_SDK，可以回调得到海康摄像头的原始PS码流。对PS码流进行解析分包，能够得到H264/H265视频流和AAC/G711音频流。然后可以将音视频通过RTMP打包封装之后发送到流媒体平台zlmediakit。因此有必要对海康PS码流格式进行分析。
1.海康PS码流：前面40位是海康私有码流头，可忽略。49 4D 4B 48 *****
2.PS码流：剩余的就是一段一段的PS包
3.PS Header：参考https://blog.csdn.net/weixin_44517656/article/details/108412988
00 00 01 BA 46 FA F7 C4 54 01 00 00 03 FE FF FF 00 00 00 01 20byte: 4byte start: 00 00 01 BA 9byte buf: 46 FA F7 C4 54 01 00 00 03 1byte stufflen: FE FE&07 = 6 6byte stuffer：FF FF 00 00 00 01
4.Program Stream Map 参考：https://www.cnblogs.com/zhumengke/articles/11681668.html
00 00 01 BC 00 5A E0 FF 00 24 40 0E 48 4B 00 00 17 9D 3E B4 60 00 00 FF FF FF 41 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 2C 1B E0 00 10 42 0E 00 00 A0 21 05 00 02 D0 12 1C 3F 00 1C 21 0F C0 00 0C 43 0A 00 00 FE 00 FA 03 01 F4 03 FF BD BD 00 00 BF BF 00 00 00 00 00 00 4byte start: 00 00 01 BC 2byte packlen: 00 5A = 90 2byte packinfo: E0 FF 2byte program_strem_info_len: 00 24 = 36 36byte descriptor: 40 0E 48 4B 00 00 17 9D 3E B4 60 00 00 FF FF FF 41 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 aa 0x40 表示当前为BASIC信息 aa+1 0x0E = 14 BASIC信息长度 aa+2~aa+3 0x48 0x4B 公司描述符 aa+6~aa+11 0x17 0x9D 0x3E 0xB4 0x60 0x00 0x00 当前年月日时分秒及加密类型 bb 0x41 表示当前为DEVICE信息 bb+1 0x12 = 18 DEVICE信息的长度 bb+4~bb+19 设备ID号 2byte elementary_stream_map_length: 00 2C = 44 1byte stream_type: 1B 0x1B --> H264 0x24 --> H265 1byte elementary_stream_id: E0 0xE0 --> 视频 0xC0 --> 音频 2byte elementary_stream_info_len: 00 10 = 16 16byte descriptor: 42 0E 00 00 A0 21 05 00 02 D0 12 1C 3F 00 1C 21 aa 0x42 表示当前为VIDEO信息 aa+1 0x0E = 14 VIDEO信息长度 aa+2~aa+3 0x00 0x00 编码器版本 aa+4~aa+5 0xA0 0x21 编码器年月日 aa+6~aa+7 0x05 0x00 = 1280 图像宽度 aa+8~aa+9 0x02 0xD0 = 720 图像高度 aa+10 0x12 是否隔行扫描,b帧数目,是否为SVC码流,是否使用e帧,最大参考帧数目 aa+11 0x1C 水印类型,显示时是否需要反隔行 aa+12 0x3F JPEG的Q值 aa+13~aa+15 0x00 0x1C 0x21 以1/90000s为单位的两帧间时间间隔,是否使用固定帧率 1byte stream_type: 0F 0x90 --> g711 0x0F --> aac 1byte elementary_stream_id: C0 0xE0 --> 视频 0xC0 --> 音频 2byte elementary_stream_info_len: 00 0C = 12 12byte descriptor: 43 0A 00 00 FE 00 FA 03 01 F4 03 FF dd 0x43 表示当前为AUDIO信息 dd+1 0x0A AUDIO信息长度 dd+2~dd+3 0x00 0x00 音频帧长度 dd+4 0xFE 音频声道数 dd+5~dd+7 0x00 0xFA 0x03 音频采样率 dd+8~dd+10 0x01 0xF4 0x03 音频比特率 2byte 私有数据信息 BD BD 4byte CRC校验 00 00 BF BF
5. PES: SPS报文 可参考：https://blog.csdn.net/smartgps2008/article/details/124130429
00 00 01 E0 00 26 8C 80 07 21 BE BD F1 15 FF FC 00 00 00 01 67 4D 00 1F 9D A8 14 01 6E 9B 80 80 80 A0 00 00 03 00 20 00 00 06 50 80 4byte start: 00 00 01 E0 2byte packlen: 00 26 = 38 2byte packinfo: 8C 80 1byte stufflen: 07 7byte stuff: 21 BE BD F1 15 FF FC 4byte start: 00 00 00 01 1byte nalType: 67 -->SPS包 23byte sps: 4D 00 1F 9D A8 14 01 6E 9B 80 80 80 A0 00 00 03 00 20 00 00 06 50 80 ss 0x4D = 77 profile_idc基准档次，66 --> baseline profile, 77 --> main profile，88 --> extended profile 后面可以解析出width、height、fps等视频信息，参见：https://github.com/cdebug/h264_to_rtmp/blob/master/src/sps_decode.h
6.PES: PPS报文
00 00 01 E0 00 0E 8C 00 03 FF FF FC 00 00 00 01 68 EE 3C 80 4byte start: 00 00 01 E0 2byte packlen：00 0E = 14 2byte packinfo: 8C 00 1byte stufflen: 03 3byte stuff: FF FF FC 4byte start: 00 00 00 01 1byte nalType: 68 -->PPS包 3byte pps: 3C 80
7.PES: I帧报文
00 00 01 E0 13 FA 8C 00 02 FF FD 00 00 00 01 65 B8 00 00 08 73 C0 AF F8 80 D7 3E AB C0 4B BF 8C A2 33 90 2B 9A 82 C2 86 98 4B E0 00 00 03 03 E7 21 45 CA 65 B8 6E 54 27 21 BE 72 DD D9 06 E2 18 C2 BB BE 58 A9 87 6F E0 31 6D 4B 3F 4D C0 9E 90 57 6A 1A 0F F2 90 13 FE D0 25 E5 A9 B9 8C 6E 67 E5 8A 05 81 7F 5D 0E EE 54 E4 B2 32 C6 47 5C 26 FE FD 48 2B B5 07 97 F5 C9 FF 82 3F 0F 5D 60 CF B0 3B 9C 6E EF 9F 5D 97 98 97 F3 C5 CF 32 AB BB 64 AB C3 C8 DE B8 A3 CF 95 08 D3 B9 BE F8 31 3F 54 C2 EE 14 3B DC C7 7F BE 94 37 FF 99 44 F4 4C 86 BD 1B 9F 89 2A EF 2E 62 99 4A CF 1B 18 26 90 ***** 4byte start: 00 00 01 E0 2byte packlen: 13 FA = 5114 2byte packinfo: 8C 00 1byte stufflen: 02 2byte stuff: FF FD 4byte start: 00 00 00 01 1byte nalType: 65 --> I帧 5105byte nalBody: B8 00 *********
8.PES: I帧报文分片
00 00 01 E0 13 FA 88 00 02 FF FF 22 DC A8 84 12 A5 18 90 DB 76 92 7C 3D AD 70 91 54 39 71 00 31 17 06 81 1C 4C 15 38 78 5C 4B F0 D8 51 B2 89 7A 92 E1 09 1E AF 67 8B 18 C8 4D DA 3C 71 E7 6D 7A B8 5B 4E EF 5B 4D 3B CB BA E8 F6 87 E3 9D E8 B8 26 C3 12 0A 41 F9 95 76 04 31 0A 64 43 D5 C0 CC CC 96 7C 77 B6 64 25 58 65 2C 8A DC 61 FD FD 1D 93 86 FB 13 30 AE A0 A4 21 31 CE 93 3C FC 6C 39 57 FA 74 93 F0 E1 8C 7E 15 CD 63 10 AD 0A C9 66 7C 7B 69 3B BD 71 46 B5 C4 AA 60 61 53 E3 01 C2 F6 AF 52 42 41 DA E1 EE 25 F9 85 3A 95 9A 5E 5F 2B 0E F5 FB CB 98 FF 8C 2D 7E C1 78 6C 7F EE CF F6 42 9C 17 F4 68 8F 1F 4E E5 60 05 F9 E2 77 28 08 39 3E 84 07 2E 77 BF FC 57 BB 2B 5E 41 35 0A ***** 4byte start: 00 00 01 E0 2byte packlen:13 FA = 5114 2byte packinfo: 88 00 1byte stufflen: 02 2byte stuff: FF FF 5109byte nalBody: 22 DC A7 ****
9.PES：P帧报文
00 00 01 E0 13 FA 8C 80 07 21 BE BF 20 31 FF FD 00 00 00 01 61 E0 08 87 7F FE 8C B3 83 77 DE EB C5 C1 BA E2 7E 08 F9 00 29 A4 2A D5 BF 03 46 60 B6 79 5E EF A3 CF 7E 85 72 9B 0F 71 36 44 06 0C 5B CB 01 95 D9 77 B4 DB BD 7D 2F 9B 89 C9 EF 55 1A 7B 7D BD 02 AC F1 DD 9D 4F 9E 36 42 4B 34 74 EF 62 C1 24 E5 F8 B5 9C 11 DD E1 CA 59 FB B6 33 93 20 6A 0B 0B 28 18 21 A2 E6 97 A7 C1 A0 8C 53 95 72 F4 9C 93 0C 41 AC 4A AD 56 9F A5 40 F9 AF 57 20 37 46 45 B8 27 B7 B9 82 36 B5 0B A3 C1 06 ***** 4byte start: 00 00 01 E0 2byte packlen: 13 FA = 5114 2byte packinfo: 8C 80 1byte stufflen: 07 2byte stuff: 21 BF BF 20 31 FF FD 4byte start: 00 00 00 01 1byte nalType: 61 --> 非I帧 5105byte nalBody: E0 08 *********
10.PES：音频报文ADTS。参考：https://blog.csdn.net/dreamInTheWorld/article/details/130600690
00 00 01 C0 00 C6 8C 80 09 2D 3E BF 77 71 FF FF FF F8 FF F9 60 40 17 5F FC 01 38 32 AC 74 26 0D 0A 04 C2 40 B1 10 2C 14 0A 84 82 81 21 88 50 22 32 0A 84 C2 23 00 00 00 00 00 EF BC A3 BF B3 FD 7D D7 B3 07 F9 C8 C8 37 60 CC 72 47 AC 43 F8 AF A0 B8 E2 F9 5B 59 F1 F2 6D CF B0 A8 90 90 A5 9B 3F E6 E7 71 25 AE 88 68 4F 0F 00 A6 38 8C D1 F4 EF 43 F4 FF AB D1 76 81 D4 3A A4 01 06 1C 7D DD 76 32 C2 7C 64 DB 8D FD 61 C5 76 2F D3 49 70 C2 7B DD 26 90 FE 1C A3 BA 3A ED 9D 5E D5 AA A5 C1 28 61 5C A3 07 33 3F 44 F1 67 2D CA C9 46 18 EE AF A4 26 62 D8 21 61 6F 70 C8 70 C5 B1 15 50 86 B9 7F 86 68 08 91 40 08 80 99 B4 07 00 00 01 BA 46 FA FC 80 C4 01 00 00 03 FE FF FF 00 00 00 02 4byte start: 00 00 01 C0 2byte packlen: 00 C6 = 198 2byte packinfo: 8C 80 1byte stufflen: 09 9byte stuff: 2D 3E BF 77 71 FF FF FF F8
ADTS_1: FF F9 60 40
12bit: syncword: FF F
1bit: id: 1001 MPEG Version: 0 --> MPEG-4; 1 --> MPEG-2
2bit: layer: 1001 always: '00'
1bit: protection+absent: 1001 标识是否进行误码校验。0 --> CRC校验; 1 --> 无校验
2bit: profile: 0110 0 --> Main profile; 1 --> Low Complexity profile(LC); 2 --> Scalable Sampling Rate profile(SSR)
4bit: sampling_frequency_index 0110 0000 0 --> 96000Hz; 4 --> 44100Hz; 8 --> 16000Hz; 11 --> 8000Hz
1bit: private_bit: 0000 私有位，编码
3bit: channel_configuration: 0000 0100 单声道
1bit: original_copy: 0100 编码时设置为0，解码时忽略
1bit: home: 0100 编码时设置为0，解码时忽略
ADTS_2: 1C DF FC
1bit： copyright_identification_bit: 0000 0001 1100 1101 1111 1111 1100 编码时设置为0，解码时忽略
1bit: copyright_identification_bit: 0000 0001 1100 1101 1111 1111 1100 编码时设置为0，解码时忽略
13bit: aac_frame_length: 0000 0001 1100 1101 1111 1111 1100 --> 230 ADTS帧长度：ADTS长度+AAC声音数据长度
11bit: adts_buffer_fullness: 0000 0001 1100 1101 1111 1111 1100 固定为0x7FF 表示是码率可变的码流
2bit: number_of_raw_data_blocks_in_frame: 0000 0001 1100 1101 1111 1111 1100 表示有number_of_raw_data_blocks_in_frame+1个原始帧
C++方式解析PS头，解析出音视频编码类型：
struct ps_head { unsigned char start_code[3]; unsigned char stream_id[1]; unsigned char marker[9]; unsigned char stuffinglen; }; struct program_stream { unsigned char start_code[3]; unsigned char stream_id[1]; unsigned short packlen; unsigned char packinfo[2]; unsigned short program_strem_info_len; }; struct elementary_stream { unsigned char stream_type[1]; unsigned char elementary_stream_id[1]; unsigned short elementary_stream_info_len; }; bool EhomeSDK::ParsePSHead(BYTE* pBuffer, int nBufLenth, AVCodecID& videoType, AVCodecID& audioType) { //unsigned char pBuffer[1024] = { // 0x00, 0x00, 0x01, 0xBA, 0x45, 0xEC, 0x36, 0x4E, 0xA4, 0x01, 0x00, 0x00, 0x03, 0xFE, 0xFF, 0xFF, // 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x01, 0xBC, 0x00, 0x5A, 0xE0, 0xFF, 0x00, 0x24, 0x40, 0x0E, // 0x48, 0x4B, 0x00, 0x00, 0x17, 0xA3, 0xB5, 0x98, 0xA0, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0x41, 0x12, // 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x00, 0x00, 0x00, 0x2C, 0x1B, 0xE0, 0x00, 0x10, 0x42, 0x0E, 0x00, 0x00, 0xA0, 0x21, 0x05, 0x00, // 0x02, 0xD0, 0x12, 0x1C, 0x3F, 0x00, 0x1C, 0x21, 0x0F, 0xC0, 0x00, 0x0C, 0x43, 0x0A, 0x00, 0x00, // 0xFE, 0x00, 0xFA, 0x03, 0x01, 0xF4, 0x03, 0xFF, 0xBD, 0xBD, 0x00, 0x00, 0xBF, 0xBF, 0x00, 0x00, // 0x00, 0x00, 0x00, 0x00 //}; videoType = audioType = AV_CODEC_ID_NONE; int index = 0; ps_head* PSHead = (ps_head*)pBuffer; PSHead->stuffinglen = PSHead->stuffinglen & 0x07; index = sizeof(ps_head) + PSHead->stuffinglen; program_stream* PSProgrm = (program_stream*)(pBuffer + index); PSProgrm->packlen = ((PSProgrm->packlen & 0x0F) << 8) + (PSProgrm->packlen >> 8); PSProgrm->program_strem_info_len = ((PSProgrm->program_strem_info_len & 0x0F) << 8) + (PSProgrm->program_strem_info_len >> 8); index = index + sizeof(program_stream) + PSProgrm->program_strem_info_len; index += 2; int elementary_stream_map_length = ((pBuffer[index] << 8) + pBuffer[index + 1]); while (index < nBufLenth) { elementary_stream* PSElementary = (elementary_stream*)(pBuffer + index); if (PSElementary->elementary_stream_id[0] == 0xE0) { if(PSElementary->stream_type[0] == 0x1B){ videoType = AV_CODEC_ID_H264; } else if(PSElementary->stream_type[0] == 0x24){ videoType = AV_CODEC_ID_H265; } } else if (PSElementary->elementary_stream_id[0] == 0xC0) { if(PSElementary->stream_type[0] == 0x0F){ videoType = AV_CODEC_ID_AAC; } else if(PSElementary->stream_type[0] == 0x90){ videoType = AV_CODEC_ID_PCM_ALAW; } audioType = PSElementary->stream_type[0]; } else { break; } PSElementary->elementary_stream_info_len = ((PSElementary->elementary_stream_info_len & 0x0F) << 8) + (PSElementary->elementary_stream_info_len >> 8); index = index + sizeof(elementary_stream) + PSElementary->elementary_stream_info_len; } return true; }
