本文章主要记录在使用gRPC时遇到的一些问题及其解决方案。

1. Method Not Found

1. 错误描述

在使用gPRC时遇到的问题。

其中客户端由golang编写,服务端由python编写。

测试时一直报如下这个错:

1
rpc error:Code=Unimplemented desc = Method Not Found

下面是官网上的错误列表,其中GRPC_STATUS_UNIMPLEMENTED对应的case也是Method not found on server

Case Status code
Client application cancelled the request GRPC_STATUS_CANCELLED
Deadline expired before server returned status GRPC_STATUS_DEADLINE_EXCEEDED
Method not found on server GRPC_STATUS_UNIMPLEMENTED
Server shutting down GRPC_STATUS_UNAVAILABLE
Server threw an exception (or did something other than returning a status code to terminate the RPC) GRPC_STATUS_UNKNOWN

但是 单独测试时 python自己写的客户端服务端可以正常交互

golang写的客户端 服务端也可以正常交互。

就是两个互相调用时会出现这个错误Method Not Found

仔细检查代码之后并没有什么问题。

各种google之后总算找到了原因。

https://www.itread01.com/content/1547029280.html

原因

这是由于.proto文件中的package name 被修改,和 server 端的package 不一致导致的,双方同步.proto文件 packagename 重新编译生成对应的代码即可。

由于python写时没有加package xxx;这就 然后go这边加了。。。怪不得会出错,果然在修改.proto文件从新编译后就能成功运行了。

2. 数据传输限制

1. 错误描述

1
details = "Received message larger than max (6194304 vs. 4194304)"

接收消息超过了最大限制(默认4M)

2. 解决

可以在建立连接的时候修改这个限制。

server

1
2
3
4
maxSize := 20 * 1024 * 1024
s := grpc.NewServer(grpc.MaxRecvMsgSize(maxSize), grpc.MaxSendMsgSize(maxSize))
PbDownMaxSize.RegisterPbDownMaxSizeServer(s, &server{})
s.Serve(listener)

client

1
2
3
maxSize := 20 * 1024 * 1024
diaOpt := grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(maxSize), grpc.MaxCallSendMsgSize(maxSize))
conn, err := grpc.Dial(endpoint, grpc.WithInsecure(), diaOpt)

3. 源码

RecvMsg

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
func (p *parser) recvMsg(maxReceiveMessageSize int) (pf payloadFormat, msg []byte, err error) {
	if _, err := p.r.Read(p.header[:]); err != nil {
		return 0, nil, err
	}

	pf = payloadFormat(p.header[0])
	length := binary.BigEndian.Uint32(p.header[1:])

	if length == 0 {
		return pf, nil, nil
	}
	if int64(length) > int64(maxInt) {
		return 0, nil, status.Errorf(codes.ResourceExhausted, "grpc: received message larger than max length allowed on current machine (%d vs. %d)", length, maxInt)
	}
	if int(length) > maxReceiveMessageSize {
		return 0, nil, status.Errorf(codes.ResourceExhausted, "grpc: received message larger than max (%d vs. %d)", length, maxReceiveMessageSize)
	}
	// TODO(bradfitz,zhaoq): garbage. reuse buffer after proto decoding instead
	// of making it for each message:
	msg = make([]byte, int(length))
	if _, err := p.r.Read(msg); err != nil {
		if err == io.EOF {
			err = io.ErrUnexpectedEOF
		}
		return 0, nil, err
	}
	return pf, msg, nil
}

可以看到在Recv的时候判定了两次数据长度。

除了设置的长度外还有一个最大长度限制int64(length) > int64(maxInt)

1
2
3
4
5
6
	if int64(length) > int64(maxInt) {
		return 0, nil, status.Errorf(codes.ResourceExhausted, "grpc: received message larger than max length allowed on current machine (%d vs. %d)", length, maxInt)
	}
	if int(length) > maxReceiveMessageSize {
		return 0, nil, status.Errorf(codes.ResourceExhausted, "grpc: received message larger than max (%d vs. %d)", length, maxReceiveMessageSize)
	}

最大限制const maxInt = int(^uint(0) >> 1)应该是2G大小。

不会有人拿gPRC传这么大的数据吧…

SendMsg

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
func (cs *clientStream) SendMsg(m interface{}) (err error) {
	defer func() {
		if err != nil && err != io.EOF {
			// Call finish on the client stream for errors generated by this SendMsg
			// call, as these indicate problems created by this client.  (Transport
			// errors are converted to an io.EOF error in csAttempt.sendMsg; the real
			// error will be returned from RecvMsg eventually in that case, or be
			// retried.)
			cs.finish(err)
		}
	}()
	if cs.sentLast {
		return status.Errorf(codes.Internal, "SendMsg called after CloseSend")
	}
	if !cs.desc.ClientStreams {
		cs.sentLast = true
	}

	// load hdr, payload, data
	hdr, payload, data, err := prepareMsg(m, cs.codec, cs.cp, cs.comp)
	if err != nil {
		return err
	}

	// TODO(dfawley): should we be checking len(data) instead?
	if len(payload) > *cs.callInfo.maxSendMessageSize {
		return status.Errorf(codes.ResourceExhausted, "trying to send message larger than max (%d vs. %d)", len(payload), *cs.callInfo.maxSendMessageSize)
	}
	msgBytes := data // Store the pointer before setting to nil. For binary logging.
	op := func(a *csAttempt) error {
		err := a.sendMsg(m, hdr, payload, data)
		// nil out the message and uncomp when replaying; they are only needed for
		// stats which is disabled for subsequent attempts.
		m, data = nil, nil
		return err
	}
	err = cs.withRetry(op, func() { cs.bufferForRetryLocked(len(hdr)+len(payload), op) })
	if cs.binlog != nil && err == nil {
		cs.binlog.Log(&binarylog.ClientMessage{
			OnClientSide: true,
			Message:      msgBytes,
		})
	}
	return
}

同理发送的时候也有这个限制

1
2
3
	if len(payload) > *cs.callInfo.maxSendMessageSize {
		return status.Errorf(codes.ResourceExhausted, "trying to send message larger than max (%d vs. %d)", len(payload), *cs.callInfo.maxSendMessageSize)
	}