TT Lab
Blog

한국어 English 日本語 中文

Get started
Blog

Blog

grpc-go Code Map: From ClientConn to http2Client, and from Serve to Handler

Status: Draft (2026-10-17), Code Reading. Baseline version: grpc/grpc-go v1.83.2 (commit 030ee8becb20, read from local Go module cache). Since the latest release is v1.84.0, line numbers may differ slightly. The 파일:줄 in the text refers to the function declaration line in v1.83.2. There are no new measurements in this part, only comparisons with the numbers from Part 6.

Table of Contents

1. One-line Summary: Company Organizational Chart

The channel is a team leader. When a task (call) is received, instead of doing it directly, the address book manager (resolver) hands over the given list to the assignment manager (balancer), and the assignment manager picks employees (subchannels) in advance and selects one using the daily assignment sheet (picker) whenever a task arrives. Each employee has their own telephone line (HTTP/2 connection). Since the assignment sheet is replaced entirely as a single paper, it does not contend for locks with anyone at the moment of picking a task.

2. Architecture Diagram

grpc-go 1.83 의 클라이언트·서버 계층 클라이언트 ClientConn clientconn.go · NewClient ccResolverWrapper → resolver resolver_wrapper.go · dns/passthrough/xds ccBalancerWrapper → balancer balancer_wrapper.go · pick_first/round_robin pickerWrapper.pick picker_wrapper.go · 호출마다 서브채널 선택 addrConn (SubConn) clientconn.go · 연결·재연결·백오프 http2Client internal/transport/http2_client.go 호출 경로: Invoke → newClientStream → pick → NewStream 서버 Server.Serve → handleRawConn server.go · 연결마다 고루틴 http2Server.operateHeaders http2_server.go · :path 해석, 스트림 생성 handleStream → processUnaryRPC server.go · 서비스 맵에서 핸들러 찾기 공용: rpc_util.go(5바이트 접두), codes·status, keepalive
Figure: grpc-go 1.83 client-server layers and files

3. Code Map

Path What Where to read first
clientconn.go Channel, subchannel (addrConn), idle mode, retry control NewClient(183), addrConn(1240)
call.go, stream.go Unary and streaming calls, retry attempts (csAttempt) invoke(65), newClientStream(213)
resolver_wrapper.go, resolver/, internal/resolver/ Resolver wrapping and dns, passthrough, and unix implementations ccResolverWrapper.UpdateState(129)
balancer_wrapper.go, balancer/ Balancer wrapping, pick_first, round_robin, ringhash, weightedroundrobin, etc. ccBalancerWrapper.NewSubConn(180)
picker_wrapper.go Selecting a subchannel per call, waiting pickerWrapper.pick(105)
internal/transport/ HTTP/2 client and server, flow control, keepalive, control buffer http2_client.go, http2_server.go
rpc_util.go 5-byte prefix, compression, receive limit, call options msgHeader(879), parser.recvMsg(785)
server.go Server, service registration, dispatch Serve(888), handleStream(1800)
codes/, status/ Status codes and status objects codes.go
health/, reflection/, channelz/, admin/ Additional services (Parts 5 and 8) health/server.go
xds/, internal/xds/ xDS resolver and balancer (Part 5) Large scale, later

4. Following the Core Path

4.1 Creating a Channel: Nothing is Connected Yet

NewClient applies options, parses the target string, selects the resolver builder, and (initParsedTargetAndResolverBuilder, default scheme dns), prepares the idle state, and finishes. Preparing the idle state (initIdleStateLocked) creates a resolver wrapper (newCCResolverWrapper) and a balancer wrapper (newCCBalancerWrapper), but the resolver does not run yet. When the first call arrives, exitIdleMode changes the channel to CONNECTING and starts the resolver. If there are no calls for 30 minutes, enterIdleMode tears down the connections with the resolver and balancer and returns to idle.

4.2 When addresses arrive: Resolver → Balancer → Subchannel → Picker

  1. When the resolver finds addresses, it calls ccResolverWrapper.UpdateState , and the channel's updateResolverStateAndUnlock applies the service config and passes it to (applyServiceConfigAndBalancer) the balancer (updateClientConnState).
  2. The balancer (e.g., round_robin) calls NewSubConn for each address, which creates newAddrConnLocked with addrConn . This addrConn is the actual entity of the subchannel.
  3. addrConn connects in the order of resetTransportAndUnlock → tryAllAddrs → createTransport , and if it fails, retries after a backoff. If it fails or gets disconnected, it requests ResolveNow from the resolver (Part 5, Section 4.1).
  4. Whenever the subchannel state changes, the balancer creates a new picker, sends it to ccBalancerWrapper.UpdateState , and pickerWrapper.updatePicker updates the picker and swaps it atomically. Calls that were waiting are awakened as the channel attached to the previous picker closes.

Why was it divided like this? By keeping address resolution (DNS, xDS, Kubernetes APIs, etc.) and selection (round-robin, weighted, hash, etc.) completely decoupled, any resolver can be combined with any balancer. Balancer updates are processed sequentially by a serializer (see the comment ingrpcsync.CallbackSerializer, balancer_wrapper.go ), and the hot path where calls pass through (pick) only reads a single atomic value, ensuring that the two worlds never wait for each other's locks.

4.3 Single Call: From Invoke to Stream

  1. The generated stub calls ClientConn.Invoke , passes through the interceptor, and invoke goes to newClientStream → newClientStreamWithParams . Even unary calls are internally implemented as a "stream of sending one message and receiving one."
  2. Creating a single attempt (newAttemptLocked), getTransport selects a subchannel from the picker. For the transport of the chosen subchannel, it sends HEADERS via csAttempt.newStream → http2Client.NewStream (Part 3's createHeaderFields).
  3. SendMsg serializes, compresses, adds a 5-byte prefix, writes to csAttempt.sendMsg , and reads the response through RecvMsg → recv → parser.recvMsg ). If it fails, withRetry looks at Part 4's shouldRetry to create a new attempt and replays the buffered sent messages.

4.4 Picker: To Wait or Fail Fast

What happens if a call arrives but there are no READY subchannels? A branch in pickerWrapper.pick provides the answer.

pickResult, err := p.Pick(info)
if err != nil {
	if err == balancer.ErrNoSubConnAvailable {
		continue
	}
	if st, ok := status.FromError(err); ok {
		...
		return pick{}, dropError{error: err}
	}
	// For all other errors, wait for ready RPCs should block and other
	// RPCs should fail with unavailable.
	if !failfast {
		lastPickErr = err
		continue
	}
	return pick{}, status.Error(codes.Unavailable, err.Error())
}

Source: grpc/grpc-go v1.83.2 picker_wrapper.go Excerpt from lines 157-176, Apache-2.0

In production, this difference looks like this: for a few seconds while restarting the server, fail fast clients spew UNAVAILABLE (like logs 10-13 of Collector in Part 6), and wait-for-ready clients quietly wait and succeed if the server comes back within the deadline.

4.5 Server: From Serve to Handler

  1. Server.Serve When Accept does handleRawConn , it spawns
  2. as a goroutine. After the credential handshake, NewServerTransport exchanges HTTP/2 settings, and serveStreams reads frames. The default maximum concurrent streams is unlimited (math.MaxUint32, server.go:189).
  3. When HEADERS arrive, operateHeaders reads :path, content-type, grpc-timeout to create a stream (invalid content-type is 415), and handleStream splits the path into service name and method name to look up in the table registered as RegisterService ( ServiceDescfrom Part 1). If not found, UNIMPLEMENTED.
  4. If unary, processUnaryRPC, if stream, processStreamingRPC reads messages, calls interceptors and handlers, and then writes trailers with writeStatus .
  5. GracefulStop sends GOAWAY to block new streams and waits for ongoing ones to finish. MaxConnectionAge in Part 5 uses the same GOAWAY.

5. Comparison with Part 6 Measurements

What was measured in Part 6 In this map
apiserver → etcd 247 TCP connections 247 etcd clients = ClientConn 247, each with one READY addrConn (since the target is a single 127.0.0.1)
242 open Watch streams on Node A Each ClientConn has one http2Client long-lived stream above it
2 kubelet → containerd channels Runtime and image ClientConn two, passthrough Unix address, not a
hubble-relay connecting to 8 nodes per node ClientConn(managed by relay's peer pool, estimated)
kubelet gRPC probes per probe ClientConn creation and termination (Part 5, Section 6)

Even with the same grpc-go, how many channels are created and how they are usedis determined by the client code, and that choice dictates the connection count, memory usage, and load balancing.

6. Where to Read First

  1. Documentation/ of keepalive.md·concurrency.md·anti-patterns.md is read first. anti-patterns.md explains why the old Dial·WithBlock should be avoided.
  2. call.go → stream.go of newClientStreamWithParams·csAttempt is used to follow a single call.
  3. picker_wrapper.go → balancer_wrapper.go → balancer/roundrobin is used for the selection side, and resolver_wrapper.go → internal/resolver/dns for the lookup side.
  4. internal/transport/http2_client.go of NewStream·reader·keepalive, controlbuf.go leads down to the write loop.
  5. For the server, reading only server.go of handleStream → processUnaryRPC reveals the dispatch structure.

7. Good First Contribution Areas

8. Limitations

Titi's note. Since the assignment table (picker) is swapped out entirely, there is no need to fight with the assignment manager for a spot even when customers rush in. Instead, if all employees fall ill, whether to wait (WaitForReady) or turn them away (fail fast) is decided by the customer!

References

All materials were accessed on 2026-10-17.

Sign in to like

Comments

No comments yet.

Sign in to leave a comment