protobuf golang&&python序列化反序列化测试

1.概要

最近考虑采用protobuf来实现kafka消息传递，所以先测试一下golang和python之前序列化互通问题。

由于go和python对于二进制的表示在ide层面是无法统一的，直接把python的字符串拷出来放到golang肯定是不行的，需要通过hex进行16进制表示法

2.bytes和hex转化语法

• Bytes -> Hex string

# Python
my_bytes = bytes([0x06, 0xe5, 0x33, 0xfd, 0x1a, 0xda, 0x86, 0x39,
                  0x1f, 0x3f, 0x6c, 0x34, 0x32, 0x04, 0xb0, 0xd2,
                  0x78, 0xd4, 0xaa, 0xec, 0x1c, 0x0b, 0x20, 0xaa,
                  0x27, 0xba, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00])



# 返回一个字符串，两个十六进制的字符表示一个byte(Return a string object containing two hexadecimal digits for each byte in the instance)
my_hex = my_bytes.hex()


// Golang

my_bytes := []byte{0x79, 0xa6, 0x1a, 0xdb, 0xc6, 0xe5, 0xa2, 0xe1,
                   0x39, 0xd2, 0x71, 0x3a, 0x54, 0x6e, 0xc7, 0xc8,
                   0x75, 0x63, 0x2e, 0x75, 0xf1, 0xdf, 0x9c, 0x3f,
                   0xa6, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}

my_hex = hex.EncodeToString(my_bytes)  // 返回十六进制编码字符串

• Hex string -> Bytes

# Python:
my_bytes = bytes.fromhex(my_hex)

//  Golang:
my_bytes = hex.DecodeString(my_hex)

 

3.实例

3.1. python protobuf输出hex字符串

python protobuf

from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
from google.protobuf import descriptor_pb2

import person_pb2 as Person

person1 = Person.Person()
person1.id = 1234
person1.name = "John Doe"
person1.email = "jdoe@example.com"
phone = person1.phones.add()  # phones字段是符合类型，调用add()方法初始化新实例。如果phones字段是标量类型，直接append()添加新元素即可。
phone.number = "555-4321"
phone.type = Person.Person.HOME

persion1Str = person1.SerializeToString()
print("persion1Str,hex", persion1Str.hex())

输出：

persion1Str,hex 0a084a6f686e20446f6510d2091a106a646f65406578616d706c652e636f6d220c0a083535352d343332311001

得到hex字符串

 

3.2. golang protobuf导入hex字符串

go protobuf

 package main

import (
	"encoding/hex"
	"google.golang.org/protobuf/proto"
	"log"
)

func main() {
	var bytes []byte

	hexstr := "0a084a6f686e20446f6510d2091a106a646f65406578616d706c652e636f6d220c0a083535352d343332311001"
	bytes, _ = hex.DecodeString(hexstr)

	person := &Person{}
    if err := proto.Unmarshal(bytes, person); err != nil {
        log.Fatalln("Failed to parse address book:", err)
    }
    fmt.Println(person)
}

输出：

name:"John Doe" id:1234 email:"jdoe@example.com" phones:{number:"555-4321" type:HOME}

测试完成