django服务使用rpc传输数据
- 1 概述
- 2 基础依赖
- 3 定义服务和消息
- 4 生成 gRPC 代码
- 5 创建服务和客户端服务
- 6 启动服务端和客户端
- 7 Django中集成gRPC
- 8 安全认证方面
- 9 健康检测
- 10 相关文档生成
概述
gRPC(gRPC Remote Procedure Call)是一种高性能、跨语言的远程过程调用(RPC)框架,由Google开发并开源。它基于HTTP/2协议进行通信,使用Protocol Buffers(protobuf)作为默认的消息编码格式。gRPC的目标是提供简单的、高效的RPC服务,使不同应用程序、服务或组件能够跨网络进行通信,而不必担心底层通信细节。
gRPC 使用 Protocol Buffers 来定义消息和服务,这意味着在通信中使用了严格的消息类型,确保了数据的一致性和完整性。
如果使用的话,是否会对当前的随意扩展的服务带来一定的限制。
基础依赖
pip install grpcio
pip install grpcio-tools
定义服务和消息
eg:
syntax = "proto3"; package job; service JobService { rpc GetJob(GetJobRequest) returns (GetJobResponse) {} } message GetJobRequest { int32 id = 1; } // 1,2 tag message Location { int32 id = 1; string value = 2; int32 region_id = 3; string region = 4; bool is_qualified = 5; } message Skill { string value = 1; bool is_qualified = 5; } message GetJobResponse { int32 id = 1; string job_title = 2; int32 job_type_id = 3; string job_type_value = 4; string job_location = 5; Location location = 6; repeated Skill skills = 7; }
生成 gRPC 代码
python -m grpc_tools.protoc -I./protos --python_out=. --pyi_out=. --grpc_python_out=. ./protos/job.proto
/protos是根目录
或者在相关服务端和客户端导入模块
job_pb2, job_pb2_grpc = grpc.protos_and_services("job.proto")
protos_and_services
函数会动态地加载 .proto
文件,运行时生成相应的Python类和方法
在一些场景中,可能更倾向于提前生成这些文件,以便能够更清晰地查看和理解由 .proto
文件生成的代码,或者因为开发、构建或部署流程更适合使用静态生成的文件。
两者之间的选择取决于具体需求和开发流程。
如果觉得动态加载更方便,可以使用grpc.protos_and_services
如果希望能够直接查看和管理生成的Python代码,可以使用grpcio-tools
生成对应的文件
创建服务和客户端服务
Server
import os from concurrent import futures import logging import grpc from rest_framework.generics import get_object_or_404 from rpc_services import job_pb2, job_pb2_grpc from jobs.models import Job from jobs.serializers import JobDetailSerializer class JobGRPCService(job_pb2_grpc.JobService): def GetJob(self, request, context): job_id = request.id print(job_id) # job = get_object_or_404( # Job, # pk=job_id, # ) # print(job) # serializer = JobDetailSerializer( # job, context={"jobseeker": None} # ) # data = serializer.data data = { "id": 167267, "job_title": "Java Developer", "job_type_id": 1, "job_type_value": "Full-time", "job_location": "bishan", "location": { "id": 162, "value": "Bishan", "region_id": 21, "region": "Central", "is_qualified": False }, "xp_lvl": { "id": 2, "key": "1_to_3_years", "value": "1 - 3 Yrs Exp", "is_qualified": False }, "skills": [ { "value": "Python", "is_qualified": False }, { "value": "Java", "is_qualified": False } ] } print(data) # ValueError: Protocol message GetJobResponse has no "xp_lvl" field. """ int32 id = 1; string job_title = 2; int32 job_type_id = 3; string job_type_value = 4; string job_location = 5; Location location = 6; repeated Skill skills = 7; """ return job_pb2.GetJobResponse( id=data["id"], job_title=data["job_title"], job_type_id=data["job_type_id"], job_type_value=data["job_type_value"], job_location=data["job_location"], location=data["location"], skills=data["skills"], ) def serve(): print("Starting serve") logging.basicConfig() port = "50051" server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) job_pb2_grpc.add_JobServiceServicer_to_server(JobGRPCService(), server) server.add_insecure_port("[::]:" + port) server.start() print("Server started, listening on " + port) server.wait_for_termination() if __name__ == "__main__": # logging.basicConfig() serve()
Client
from __future__ import print_function
import logging
import grpc
import job_pb2
import job_pb2_grpc
from google.protobuf.json_format import MessageToJson
def run():
# NOTE(gRPC Python Team): .close() is possible on a channel and should be
# used in circumstances in which the with statement does not fit the needs
# of the code.
print("Will try to get job ...")
with grpc.insecure_channel("localhost:50051") as channel:
stub = job_pb2_grpc.JobServiceStub(channel)
response = stub.GetJob(job_pb2.GetJobRequest(id=167267))
print(f"Client received: {response}")
d = MessageToJson(response)
print(d)
if __name__ == "__main__":
logging.basicConfig()
run()
运行rpc文件.proto 生成相应的文件就可以通信了
如图:
启动服务端和客户端
python server.py python client.py
Django中集成gRPC
可以使用 django-grpc-framework
AWS Lambda部署存在一定困难 应该无法依赖WSGI运行
通过Django Command 部署运行
独立rpc服务,通过容器镜像封装
安全认证方面
https://grpc.io/docs/guides/auth/
基于 TLS/SSL 的证书认证
基于令牌的认证
HTTP 标头认证
eg:
token ="token"metadata =[('authorization','Bearer '+ token)]response = stub.GetJob(job_pb2.GetJobRequest(id=167267), metadata=metadata)
健康检测
https://github.com/grpc/grpc/blob/master/examples/python/health_checking/greeter_server.py
相关文档生成
https://github.com/pseudomuto/protoc-gen-doc