Protocol Buffers学习教程

最近看公司代码的过程中，看到了很多proto后缀的文件，这是个啥玩意？问了大佬，原来这是Protocol Buffers！

这玩意是干啥的？查完资料才知道，又是谷歌大佬推的开源组件，这玩意完全可以取代XML和JSON的数据交换格式，而且更加快！

Protocol Buffer 即 PB 是大 Google 公司推行的一套混合语言数据标准, 标准介绍如下:

是 Google 开源的一种轻便高效的结构化数据存储格式，可以用于结构化数据的串行化，也称作序列化，主要用于数据存储或是 RPC 数据交换，支持多语言，可拓展. 它很适合做数据存储或 RPC 数据交换格式. 可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。

感觉是不是比较抽象?

简单而言, 它的出现就是想打败 xml & json, pb 其实和他俩差不多。

既然是 Google 推行的, 肯定是采用开源策略, 附上 github 地址 (目前 star 超过 2W)

PB 首页地址

优点

◇性能好 / 效率高
◇代码生成机制 – 这个后面说
◇支持 “向后兼容” 和 “向前兼容”
◇支持多种编程语言

缺点

◇二进制格式导致可读性差
为了提高性能，protobuf 采用了二进制格式进行编码。这直接导致了可读性差的问题（严格地说，是没有可读性）
◇缺乏自描述
一般来说，XML 是自描述的，而 protobuf 格式则不是。给你一段二进制格式的协议内容，如果不配合相应的 proto 文件，那简直就像天书一般。

参考下面这篇文章安装：

mac 安装 protocol buffer（2.6.1 版）教程

先看一下简介：

https://developers.google.cn/protocol-buffers/docs/overview

直接上手：

https://developers.google.cn/protocol-buffers/docs/tutorials

我最近在学Go，这里就用Go了，你只需要选择自己感兴趣的语言即可。

首先看一下我的项目目录：

在$GOPATH/src/github.com/chenchi/procotolbuffers/这个目录。

开始我写一个addressbook.proto的文件，具体里面的字段啥含义，直接看上面的文档，虽然是英文，很容易理解。

syntax = "proto3";
package procotolbuffers;

import "google/protobuf/timestamp.proto";

message Person {
    string name = 1;
    int32 id = 2;  // Unique ID number for this person.
    string email = 3;

    enum PhoneType {
        MOBILE = 0;
        HOME = 1;
        WORK = 2;
    }

    message PhoneNumber {
        string number = 1;
        PhoneType type = 2;
    }

    repeated PhoneNumber phones = 4;

    google.protobuf.Timestamp last_updated = 5;
}

// Our address book file is just one of these.
message AddressBook {
    repeated Person people = 1;
}

接着执行：

go get -u github.com/golang/protobuf/protoc-gen-go

protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto

我打算跟addressbook.proto生成在同一目录下，所以我就写当前目录了，即

protoc -I=. --go_out=. ./addressbook.proto

生成完了就是那个addressbook.pb.go文件，这里面很多东西：

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: addressbook.proto

package procotolbuffers

import (
    fmt "fmt"
    proto "github.com/golang/protobuf/proto"
    timestamp "github.com/golang/protobuf/ptypes/timestamp"
    math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package

type Person_PhoneType int32

const (
    Person_MOBILE Person_PhoneType = 0
    Person_HOME   Person_PhoneType = 1
    Person_WORK   Person_PhoneType = 2
)

var Person_PhoneType_name = map[int32]string{
    0: "MOBILE",
    1: "HOME",
    2: "WORK",
}

var Person_PhoneType_value = map[string]int32{
    "MOBILE": 0,
    "HOME":   1,
    "WORK":   2,
}

func (x Person_PhoneType) String() string {
    return proto.EnumName(Person_PhoneType_name, int32(x))
}

func (Person_PhoneType) EnumDescriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{0, 0}
}

type Person struct {
    Name                 string                `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
    Id                   int32                 `protobuf:"varint,2,opt,name=id,proto3" json:"id,omitempty"`
    Email                string                `protobuf:"bytes,3,opt,name=email,proto3" json:"email,omitempty"`
    Phones               []*Person_PhoneNumber `protobuf:"bytes,4,rep,name=phones,proto3" json:"phones,omitempty"`
    LastUpdated          *timestamp.Timestamp  `protobuf:"bytes,5,opt,name=last_updated,json=lastUpdated,proto3" json:"last_updated,omitempty"`
    XXX_NoUnkeyedLiteral struct{}              `json:"-"`
    XXX_unrecognized     []byte                `json:"-"`
    XXX_sizecache        int32                 `json:"-"`
}

func (m *Person) Reset()         { *m = Person{} }
func (m *Person) String() string { return proto.CompactTextString(m) }
func (*Person) ProtoMessage()    {}
func (*Person) Descriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{0}
}

func (m *Person) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_Person.Unmarshal(m, b)
}
func (m *Person) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_Person.Marshal(b, m, deterministic)
}
func (m *Person) XXX_Merge(src proto.Message) {
    xxx_messageInfo_Person.Merge(m, src)
}
func (m *Person) XXX_Size() int {
    return xxx_messageInfo_Person.Size(m)
}
func (m *Person) XXX_DiscardUnknown() {
    xxx_messageInfo_Person.DiscardUnknown(m)
}

var xxx_messageInfo_Person proto.InternalMessageInfo

func (m *Person) GetName() string {
    if m != nil {
        return m.Name
    }
    return ""
}

func (m *Person) GetId() int32 {
    if m != nil {
        return m.Id
    }
    return 0
}

func (m *Person) GetEmail() string {
    if m != nil {
        return m.Email
    }
    return ""
}

func (m *Person) GetPhones() []*Person_PhoneNumber {
    if m != nil {
        return m.Phones
    }
    return nil
}

func (m *Person) GetLastUpdated() *timestamp.Timestamp {
    if m != nil {
        return m.LastUpdated
    }
    return nil
}

type Person_PhoneNumber struct {
    Number               string           `protobuf:"bytes,1,opt,name=number,proto3" json:"number,omitempty"`
    Type                 Person_PhoneType `protobuf:"varint,2,opt,name=type,proto3,enum=procotolbuffers.Person_PhoneType" json:"type,omitempty"`
    XXX_NoUnkeyedLiteral struct{}         `json:"-"`
    XXX_unrecognized     []byte           `json:"-"`
    XXX_sizecache        int32            `json:"-"`
}

func (m *Person_PhoneNumber) Reset()         { *m = Person_PhoneNumber{} }
func (m *Person_PhoneNumber) String() string { return proto.CompactTextString(m) }
func (*Person_PhoneNumber) ProtoMessage()    {}
func (*Person_PhoneNumber) Descriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{0, 0}
}

func (m *Person_PhoneNumber) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_Person_PhoneNumber.Unmarshal(m, b)
}
func (m *Person_PhoneNumber) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_Person_PhoneNumber.Marshal(b, m, deterministic)
}
func (m *Person_PhoneNumber) XXX_Merge(src proto.Message) {
    xxx_messageInfo_Person_PhoneNumber.Merge(m, src)
}
func (m *Person_PhoneNumber) XXX_Size() int {
    return xxx_messageInfo_Person_PhoneNumber.Size(m)
}
func (m *Person_PhoneNumber) XXX_DiscardUnknown() {
    xxx_messageInfo_Person_PhoneNumber.DiscardUnknown(m)
}

var xxx_messageInfo_Person_PhoneNumber proto.InternalMessageInfo

func (m *Person_PhoneNumber) GetNumber() string {
    if m != nil {
        return m.Number
    }
    return ""
}

func (m *Person_PhoneNumber) GetType() Person_PhoneType {
    if m != nil {
        return m.Type
    }
    return Person_MOBILE
}

// Our address book file is just one of these.
type AddressBook struct {
    People               []*Person `protobuf:"bytes,1,rep,name=people,proto3" json:"people,omitempty"`
    XXX_NoUnkeyedLiteral struct{}  `json:"-"`
    XXX_unrecognized     []byte    `json:"-"`
    XXX_sizecache        int32     `json:"-"`
}

func (m *AddressBook) Reset()         { *m = AddressBook{} }
func (m *AddressBook) String() string { return proto.CompactTextString(m) }
func (*AddressBook) ProtoMessage()    {}
func (*AddressBook) Descriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{1}
}

func (m *AddressBook) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_AddressBook.Unmarshal(m, b)
}
func (m *AddressBook) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_AddressBook.Marshal(b, m, deterministic)
}
func (m *AddressBook) XXX_Merge(src proto.Message) {
    xxx_messageInfo_AddressBook.Merge(m, src)
}
func (m *AddressBook) XXX_Size() int {
    return xxx_messageInfo_AddressBook.Size(m)
}
func (m *AddressBook) XXX_DiscardUnknown() {
    xxx_messageInfo_AddressBook.DiscardUnknown(m)
}

var xxx_messageInfo_AddressBook proto.InternalMessageInfo

func (m *AddressBook) GetPeople() []*Person {
    if m != nil {
        return m.People
    }
    return nil
}

func init() {
    proto.RegisterEnum("procotolbuffers.Person_PhoneType", Person_PhoneType_name, Person_PhoneType_value)
    proto.RegisterType((*Person)(nil), "procotolbuffers.Person")
    proto.RegisterType((*Person_PhoneNumber)(nil), "procotolbuffers.Person.PhoneNumber")
    proto.RegisterType((*AddressBook)(nil), "procotolbuffers.AddressBook")
}

func init() { proto.RegisterFile("addressbook.proto", fileDescriptor_1eb1a68c9dd6d429) }

var fileDescriptor_1eb1a68c9dd6d429 = []byte{
    // 321 bytes of a gzipped FileDescriptorProto
    0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x7c, 0x91, 0x41, 0x4b, 0xc3, 0x40,
    0x10, 0x85, 0x4d, 0x9a, 0x06, 0x3b, 0x91, 0x5a, 0x17, 0xd1, 0xd0, 0x8b, 0xb1, 0x5e, 0x02, 0x42,
    0x0a, 0x15, 0x4f, 0xa2, 0x60, 0xa1, 0xa0, 0x68, 0x6d, 0x59, 0x2a, 0x5e, 0x04, 0x49, 0xcc, 0xb4,
    0x86, 0x26, 0x99, 0x25, 0xbb, 0x39, 0xf4, 0x27, 0xfa, 0xaf, 0x24, 0x9b, 0x54, 0x44, 0xd0, 0xdb,
    0x9b, 0xdd, 0x6f, 0x76, 0xde, 0x9b, 0x85, 0x83, 0x30, 0x8e, 0x0b, 0x94, 0x32, 0x22, 0x5a, 0x07,
    0xa2, 0x20, 0x45, 0x6c, 0x5f, 0x14, 0xf4, 0x4e, 0x8a, 0xd2, 0xa8, 0x5c, 0x2e, 0xb1, 0x90, 0xfd,
    0x93, 0x15, 0xd1, 0x2a, 0xc5, 0xa1, 0xbe, 0x8e, 0xca, 0xe5, 0x50, 0x25, 0x19, 0x4a, 0x15, 0x66,
    0xa2, 0xee, 0x18, 0x7c, 0x9a, 0x60, 0xcf, 0xb1, 0x90, 0x94, 0x33, 0x06, 0x56, 0x1e, 0x66, 0xe8,
    0x1a, 0x9e, 0xe1, 0x77, 0xb8, 0xd6, 0xac, 0x0b, 0x66, 0x12, 0xbb, 0xa6, 0x67, 0xf8, 0x6d, 0x6e,
    0x26, 0x31, 0x3b, 0x84, 0x36, 0x66, 0x61, 0x92, 0xba, 0x2d, 0x0d, 0xd5, 0x05, 0xbb, 0x02, 0x5b,
    0x7c, 0x50, 0x8e, 0xd2, 0xb5, 0xbc, 0x96, 0xef, 0x8c, 0xce, 0x82, 0x5f, 0x3e, 0x82, 0x7a, 0x44,
    0x30, 0xaf, 0xa8, 0xa7, 0x32, 0x8b, 0xb0, 0xe0, 0x4d, 0x0b, 0xbb, 0x86, 0xbd, 0x34, 0x94, 0xea,
    0xad, 0x14, 0x71, 0xa8, 0x30, 0x76, 0xdb, 0x9e, 0xe1, 0x3b, 0xa3, 0x7e, 0x50, 0x3b, 0x0f, 0xb6,
    0xce, 0x83, 0xc5, 0xd6, 0x39, 0x77, 0x2a, 0xfe, 0xb9, 0xc6, 0xfb, 0xaf, 0xe0, 0xfc, 0x78, 0x95,
    0x1d, 0x81, 0x9d, 0x6b, 0xd5, 0xc4, 0x68, 0x2a, 0x76, 0x09, 0x96, 0xda, 0x08, 0xd4, 0x51, 0xba,
    0xa3, 0xd3, 0x7f, 0x0d, 0x2e, 0x36, 0x02, 0xb9, 0xc6, 0x07, 0xe7, 0xd0, 0xf9, 0x3e, 0x62, 0x00,
    0xf6, 0x74, 0x36, 0xbe, 0x7f, 0x9c, 0xf4, 0x76, 0xd8, 0x2e, 0x58, 0x77, 0xb3, 0xe9, 0xa4, 0x67,
    0x54, 0xea, 0x65, 0xc6, 0x1f, 0x7a, 0xe6, 0xe0, 0x06, 0x9c, 0xdb, 0xfa, 0x4b, 0xc6, 0x44, 0x6b,
    0x36, 0x04, 0x5b, 0x20, 0x89, 0xb4, 0xda, 0x68, 0xb5, 0x95, 0xe3, 0x3f, 0x86, 0xf2, 0x06, 0x8b,
    0x6c, 0x9d, 0xf5, 0xe2, 0x2b, 0x00, 0x00, 0xff, 0xff, 0xba, 0x00, 0x86, 0x05, 0xd9, 0x01, 0x00,
    0x00,
}

里面的struct实现了proto.Message接口。

验证一下，我写了个main.go方法，

验证写：

package main

import (
    pb "github.com/chenchi/procotolbuffers"
    "github.com/gogo/protobuf/proto"
    "io/ioutil"
    "log"
)

func main() {
    p := &pb.Person{
        Id:    1234,
        Name:  "chenchi",
        Email: "chenchi@example.com",
        Phones: []*pb.Person_PhoneNumber{
            {Number: "555-4321", Type: pb.Person_HOME},
        },
    }
    
    //写
    out, err := proto.Marshal(p)
    if err != nil {
        log.Fatalln("Failed to encode address person:", err)
    }
    if err := ioutil.WriteFile("github.com/chenchi/procotolbuffers/main/out", out, 0644); err != nil {
        log.Fatalln("Failed to write address person:", err)
    }
    
}

执行完生成了out文件。成功！

验证读：

package main

import (
    "fmt"
    pb "github.com/chenchi/procotolbuffers"
    "github.com/gogo/protobuf/proto"
    "io/ioutil"
    "log"
)

func main() {
    
    //读
    in, err := ioutil.ReadFile("github.com/chenchi/procotolbuffers/main/out")
    if err != nil {
        log.Fatalln("Error reading file:", err)
    }
    p2 := &pb.Person{}
    if err := proto.Unmarshal(in, p2); err != nil {
        log.Fatalln("Failed to parse address book:", err)
    }
    fmt.Println(p2)

}