trpc.group/trpc-go/trpc-go@v1.0.3/http/README.md (about) 1 English | [中文](README.zh_CN.md) 2 3 # tRPC-Go HTTP protocol 4 5 The tRPC-Go framework supports building three types of HTTP-related services: 6 7 1. pan-HTTP standard service (no stub code and IDL file required) 8 2. pan-HTTP RPC service (shares the stub code and IDL files used by the RPC protocol) 9 3. pan-HTTP RESTful service (provides RESTful API based on IDL and stub code) 10 11 The RESTful related documentation is available in [/restful](/restful/) 12 13 ## Pan-HTTP standard services 14 15 The tRPC-Go framework provides pervasive HTTP standard service capabilities, mainly by adding service registration, service discovery, interceptors and other capabilities to the annotation library HTTP, so that the HTTP protocol can be seamlessly integrated into the tRPC ecosystem 16 17 Compared with the tRPC protocol, the pan-HTTP standard service service does not rely on stub code, so the protocol on the service side is named `http_no_protocol`. 18 19 ### Server-side 20 21 #### configuration writing 22 23 Configure the service in the `trpc_go.yaml` configuration file with protocol `http_no_protocol` and http2 with `http2_no_protocol`: 24 25 ```yaml 26 server: 27 service: # The service provided by the business service, there can be more than one 28 - name: trpc.app.server.stdhttp # The service's route name 29 network: tcp # the type of network listening, tcp or udp 30 protocol: http_no_protocol # Application layer protocol http_no_protocol 31 timeout: 1000 # Maximum request processing time, in milliseconds 32 ip: 127.0.0.1 33 port: 8080 # Service listening port 34 ``` 35 36 Take care to ensure that the configuration file is loaded properly 37 38 #### code writing 39 40 ##### single URL registration 41 42 ```go 43 import ( 44 "net/http" 45 46 "trpc.group/trpc-go/trpc-go/codec" 47 "trpc.group/trpc-go/trpc-go/log" 48 thttp "trpc.group/trpc-go/trpc-go/http" 49 trpc "trpc.group/trpc-go/trpc-go" 50 ) 51 52 func main() { 53 s := trpc.NewServer() 54 thttp.HandleFunc("/xxx", handle) 55 // The parameters passed when registering the NoProtocolService must match the service name in the configuration: s.Service("trpc.app.server.stdhttp") 56 thttp.RegisterNoProtocolService(s.Service("trpc.app.server.stdhttp")) 57 s.Serve() 58 } 59 60 func handle(w http.ResponseWriter, r *http.Request) error { 61 // handle is written in exactly the same way as the standard library HTTP 62 // For example, you can read the header in r, etc. 63 // You can stream the packet to the client in r. 64 clientReq, err := io.ReadAll(r.Body) 65 if err ! = nil { /*... */ } 66 // Finally use w for packet return 67 w.Header().Set("Content-type", "application/text") 68 w.WriteHeader(http.StatusOK) 69 w.Write([]byte("response body")) 70 return nil 71 } 72 ``` 73 74 ##### MUX Registration 75 76 ```go 77 import ( 78 "net/http" 79 80 "trpc.group/trpc-go/trpc-go/codec" 81 "trpc.group/trpc-go/trpc-go/log" 82 thttp "trpc.group/trpc-go/trpc-go/http" 83 trpc "trpc.group/trpc-go/trpc-go" 84 "github.com/gorilla/mux" 85 ) 86 87 func main() { 88 s := trpc.NewServer() 89 // Routing registration 90 router := mux.NewRouter() 91 router.HandleFunc("/{dir0}/{dir1}/{day}/{hour}/{vid:[a-z0-9A-Z]+}_{index:[0-9]+}.jpg", handle). 92 Methods("GET") 93 // The parameters passed when registering RegisterNoProtocolServiceMux must be consistent with the service name in the configuration: s.Service("trpc.app.server.stdhttp") 94 thttp.RegisterNoProtocolServiceMux(s.Service("trpc.app.server.stdhttp"), router) 95 s.Serve() 96 } 97 98 func handle(w http.ResponseWriter, r *http.Request) error { 99 // take the arguments in the url 100 vars := mux.Vars(r) 101 vid := vars["vid"] 102 index := vars["index"] 103 log.Infof("vid: %s, index: %s", vid, index) 104 return nil 105 } 106 ``` 107 108 ### Client 109 110 This refers to calling a standard HTTP service, which is not necessarily built on the tRPC-Go framework downstream 111 112 The cleanest way is actually to use the HTTP Client provided by the standard library directly, but you can't use the service discovery and various plug-in interceptors that provide capabilities (such as monitoring reporting) 113 114 #### configuration writing 115 116 ```yaml 117 client: # backend configuration for client calls 118 timeout: 1000 # Maximum processing time for all backend requests 119 namespace: Development # environment for all backends 120 filter: # List of interceptors before and after all backend function calls 121 - simpledebuglog # This is the debug log interceptor, you can add other interceptors, such as monitoring, etc. 122 service: # Configuration for a single backend 123 - name: trpc.app.server.stdhttp # service name of the downstream http service 124 # # You can use target to select other selector, only service name will be used for service discovery by default (in case of using polaris plugin) 125 # target: polaris://trpc.app.server.stdhttp # or ip://127.0.0.1:8080 to specify ip:port for invocation 126 ``` 127 128 #### code writing 129 130 ```go 131 package main 132 133 import ( 134 "context" 135 136 trpc "trpc.group/trpc-go/trpc-go" 137 "trpc.group/trpc-go/trpc-go/client" 138 "trpc.group/trpc-go/trpc-go/codec" 139 "trpc.group/trpc-go/trpc-go/http" 140 "trpc.group/trpc-go/trpc-go/log" 141 ) 142 143 // Data is request message data. 144 type Data struct { 145 Msg string 146 } 147 148 func main() { 149 // Omit the tRPC-Go framework configuration loading part, if the following logic is in an RPC handle, 150 // the configuration has generally been loaded normally. 151 // Create ClientProxy, set the protocol to HTTP protocol, and serialize it to JSON. 152 httpCli := http.NewClientProxy("trpc.app.server.stdhttp", 153 client.WithSerializationType(codec.SerializationTypeJSON)) 154 reqHeader := &http.ClientReqHeader{} 155 // Add request field for HTTP Head. 156 reqHeader.AddHeader("request", "test") 157 rspHead := &http.ClientRspHeader{} 158 req := &Data{Msg: "Hello, I am stdhttp client!"} 159 rsp := &Data{} 160 // Send HTTP POST request. 161 if err := httpCli.Post(context.Background(), "/v1/hello", req, rsp, 162 client.WithReqHead(reqHeader), 163 client.WithRspHead(rspHead), 164 ); err != nil { 165 log.Warn("get http response err") 166 return 167 } 168 // Get the reply field in the HTTP response header. 169 replyHead := rspHead.Response.Header.Get("reply") 170 log.Infof("data is %s, request head is %s\n", rsp, replyHead) 171 } 172 ``` 173 174 ## Pan HTTP RPC Service 175 176 Compared to the **Pan HTTP Standard Service**, the main difference of the Pan HTTP RPC Service is the reuse of the IDL protocol file and its generated stub code, while seamlessly integrating into the tRPC ecosystem (service registration, service routing, service discovery, various plug-in interceptors, etc.) 177 178 Note: 179 180 In this service form, the HTTP protocol is consistent with the tRPC protocol: when the server returns a failure, the body is empty and the error code error message is placed in the HTTP header 181 182 ### Server-side 183 184 #### configuration writing 185 186 First you need to generate the stub code: 187 188 ```shell 189 trpc create -p helloworld.proto --protocol http -o out 190 ``` 191 192 If you are already a tRPC service and want to support the HTTP protocol on the same interface, you don't need to generate the stakes again, just add the `http` protocol to the configuration 193 194 ```yaml 195 server: # server-side configuration 196 service: 197 ## The same interface can provide both trpc protocol and http protocol services through two configurations 198 - name: trpc.test.helloworld.Greeter # service's route name 199 ip: 127.0.0.0 # service listener ip address can use placeholder ${ip},ip or nic, ip is preferred 200 port: 80 # The service listens to the port. 201 protocol: trpc # Application layer protocol trpc http 202 ## Here is the main example, note that the application layer protocol is http 203 - name: trpc.test.helloworld.GreeterHTTP # service's route name 204 ip: 127.0.0.0 # service listener ip address can use placeholder ${ip},ip or nic, ip is preferred 205 port: 80 # The service listens to the port. 206 protocol: http # Application layer protocol trpc http 207 ``` 208 209 #### code writing 210 211 ```go 212 import ( 213 "context" 214 "fmt" 215 216 trpc "trpc.group/trpc-go/trpc-go" 217 "trpc.group/trpc-go/trpc-go/client" 218 pb "github.com/xxxx/helloworld/pb" 219 ) 220 221 func main() { 222 s := trpc.NewServer() 223 hello := Hello{} 224 pb.RegisterHelloTrpcGoService(s.Service("trpc.test.helloworld.Greeter"), &hello) 225 // Same as the normal tRPC service registration 226 pb.RegisterHelloTrpcGoService(s.Service("trpc.test.helloworld.GreeterHTTP"), &hello) 227 log.Println(s.Serve()) 228 } 229 230 type Hello struct {} 231 232 // The implementation of the RPC service interface does not need to be aware of the HTTP protocol, it just needs to follow the usual logic to process the request and return a response 233 func (h *Hello) Hello(ctx context.Context, req *pb.HelloReq) (*pb.HelloRsp, error) { 234 fmt.Println("--- got HelloReq", req) 235 time.Sleep(time.Second) 236 return &pb.HelloRsp{Msg: "Welcome " + req.Name}, nil 237 } 238 ``` 239 #### Custom URL path 240 241 Default is `/package.service/method`, you can customize any URL by alias parameter 242 243 - Protocol definition. 244 245 ```protobuf 246 syntax = "proto3"; 247 package trpc.app.server; 248 option go_package="github.com/your_repo/app/server"; 249 250 import "trpc.proto"; 251 252 message Request { 253 bytes req = 1; 254 } 255 256 message Reply { 257 bytes rsp = 1; 258 } 259 260 service Greeter { 261 rpc SayHello(Request) returns (Reply) { 262 option (trpc.alias) = "/cgi-bin/module/say_hello"; 263 }; 264 } 265 ``` 266 267 #### Custom error code handling functions 268 269 The default error handling function, which populates the `trpc-ret/trpc-func-ret` field in the HTTP header, can also be replaced by defining your own ErrorHandler. 270 271 ```golang 272 import ( 273 "net/http" 274 275 "trpc.group/trpc-go/trpc-go/errs" 276 thttp "trpc.group/trpc-go/trpc-go/http" 277 ) 278 279 func init() { 280 thttp.DefaultServerCodec.ErrHandler = func(w http.ResponseWriter, r *http.Request, e *errs.Error) { 281 // Generally define your own retcode retmsg field, compose the json and write it to the response body 282 w.Write([]byte(fmt.Sprintf(`{"retcode":%d, "retmsg":"%s"}`, e.Code, e.Msg))) 283 // Each business team can define it in their own git, and the business code can be imported into it 284 } 285 } 286 ``` 287 288 289 ### Client 290 291 There is considerable flexibility in actually calling a pan-HTTP RPC service, as the service provides the HTTP protocol externally, so any HTTP Client can be called, in general, in one of three ways: 292 293 * using the standard library HTTP Client, which constructs the request and parses the response based on the interface documentation provided downstream, with the disadvantage that it does not fit into the tRPC ecosystem (service discovery, plug-in interceptors, etc.) 294 * `NewStdHTTPClient`, which constructs requests and parses responses based on downstream documentation, can be integrated into the tRPC ecosystem, but request responses require documentation to construct and parse. 295 * `NewClientProxy`, using `Get/Post/Put` interfaces on top of the returned `Client`, can be integrated into the tRPC ecosystem, and `req,rsp` strictly conforms to the definition in the IDL protocol file, can reuse the stub code, the disadvantage is the lack of flexibility of the standard library HTTP Client, For example, it is not possible to read back packets in a stream 296 297 `NewStdHTTPClient` is used in the **client** section of the **Pan HTTP Standard Service**, and the following describes the stub-based HTTP Client `thttp.NewClientProxy`. 298 299 #### configuration writing 300 301 It is written in the same way as a normal RPC Client, just change the configuration `protocol` to `http`: 302 303 ```yaml 304 client: 305 namespace: Development # for all backend environments 306 filter: # List of interceptors for all backends before and after function calls 307 service: # Configuration for a single backend 308 - name: trpc.test.helloworld.GreeterHTTP # service name of the backend service 309 network: tcp # The network type of the backend service tcp udp 310 protocol: http # Application layer protocol trpc http 311 # # You can use target to select other selector, only service name will be used by default for service discovery (if Polaris plugin is used) 312 # target: ip://127.0.0.1:8000 # request service address 313 timeout: 1000 # maximum request processing time 314 ``` 315 316 #### code writing 317 318 ```go 319 // Package main is the main package. 320 package main 321 import ( 322 "context" 323 "net/http" 324 325 "trpc.group/trpc-go/trpc-go/client" 326 thttp "trpc.group/trpc-go/trpc-go/http" 327 "trpc.group/trpc-go/trpc-go/log" 328 pb "trpc.group/trpc-go/trpc-go/testdata/trpc/helloworld" 329 ) 330 func main() { 331 // omit the configuration loading part of the tRPC-Go framework, if the following logic is in some RPC handle, the configuration is usually already loaded properly 332 // Create a ClientProxy, set the protocol to HTTP, serialize it to JSON 333 proxy := pb.NewGreeterClientProxy() 334 reqHeader := &thttp.ClientReqHeader{} 335 // must be left blank or set to "POST" 336 reqHeader.Method = "POST" 337 // Add request field to HTTP Head 338 reqHeader.AddHeader("request", "test") 339 // Set a cookie 340 cookie := &http.Cookie{Name: "sample", Value: "sample", HttpOnly: false} 341 reqHeader.AddHeader("Cookie", cookie.String()) 342 req := &pb.HelloRequest{Msg: "Hello, I am tRPC-Go client."} 343 rspHead := &thttp.ClientRspHeader{} 344 // Send HTTP RPC request 345 rsp, err := proxy.SayHello(context.Background(), req, 346 client.WithReqHead(reqHeader), 347 client.WithRspHead(rspHead), 348 // Here you can use the code to force the target field in trpc_go.yaml to be overridden to set other selectors, which is generally not necessary, this is just a demonstration of the functionality 349 // client.WithTarget("ip://127.0.0.1:8000"), 350 ) 351 if err != nil { 352 log.Warn("get http response err") 353 return 354 } 355 // Get the reply field in the HTTP response header 356 replyHead := rspHead.Response.Header.Get("reply") 357 log.Infof("data is %s, request head is %s\n", rsp, replyHead) 358 } 359 ``` 360 361 ## FAQ 362 363 ### Enable HTTPS for Client and Server 364 365 #### Mutual Authentication 366 367 ##### Configuration Only 368 369 Simply add the corresponding configuration items (certificate and private key) in `trpc_go.yaml`: 370 371 ```yaml 372 server: # Server configuration 373 service: # Business services provided, can have multiple 374 - name: trpc.app.server.stdhttp 375 network: tcp 376 protocol: http_no_protocol # Fill in http for generic HTTP RPC services 377 tls_cert: "../testdata/server.crt" # Add certificate path 378 tls_key: "../testdata/server.key" # Add private key path 379 ca_cert: "../testdata/ca.pem" # CA certificate, fill in when mutual authentication is required 380 client: # Client configuration 381 service: # Business services provided, can have multiple 382 - name: trpc.app.server.stdhttp 383 network: tcp 384 protocol: http 385 tls_cert: "../testdata/server.crt" # Add certificate path 386 tls_key: "../testdata/server.key" # Add private key path 387 ca_cert: "../testdata/ca.pem" # CA certificate, fill in when mutual authentication is required 388 ``` 389 390 No additional TLS/HTTPS-related operations are needed in the code (no need to specify the scheme as `https`, no need to manually add the `WithTLS` option, and no need to find a way to include an HTTPS-related identifier in `WithTarget` or other places). 391 392 ##### Code Only 393 394 For the server, use `server.WithTLS` to specify the server certificate, private key, and CA certificate in order: 395 396 ```go 397 server.WithTLS( 398 "../testdata/server.crt", 399 "../testdata/server.key", 400 "../testdata/ca.pem", 401 ) 402 ``` 403 404 For the client, use `client.WithTLS` to specify the client certificate, private key, CA certificate, and server name in order: 405 406 ```go 407 client.WithTLS( 408 "../testdata/client.crt", 409 "../testdata/client.key", 410 "../testdata/ca.pem", 411 "localhost", // Fill in the server name 412 ) 413 ``` 414 415 No additional TLS/HTTPS-related operations are needed in the code. 416 417 Example: 418 419 ```go 420 func TestHTTPSUseClientVerify(t *testing.T) { 421 const ( 422 network = "tcp" 423 address = "127.0.0.1:0" 424 ) 425 ln, err := net.Listen(network, address) 426 require.Nil(t, err) 427 defer ln.Close() 428 serviceName := "trpc.app.server.Service" + t.Name() 429 service := server.New( 430 server.WithServiceName(serviceName), 431 server.WithNetwork("tcp"), 432 server.WithProtocol("http_no_protocol"), 433 server.WithListener(ln), 434 server.WithTLS( 435 "../testdata/server.crt", 436 "../testdata/server.key", 437 "../testdata/ca.pem", 438 ), 439 ) 440 thttp.HandleFunc("/", func(w http.ResponseWriter, _ *http.Request) error { 441 w.Write([]byte(t.Name())) 442 return nil 443 }) 444 thttp.RegisterNoProtocolService(service) 445 s := &server.Server{} 446 s.AddService(serviceName, service) 447 go s.Serve() 448 defer s.Close(nil) 449 time.Sleep(100 * time.Millisecond) 450 451 c := thttp.NewClientProxy( 452 serviceName, 453 client.WithTarget("ip://"+ln.Addr().String()), 454 ) 455 req := &codec.Body{} 456 rsp := &codec.Body{} 457 require.Nil(t, 458 c.Post(context.Background(), "/", req, rsp, 459 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 460 client.WithSerializationType(codec.SerializationTypeNoop), 461 client.WithCurrentCompressType(codec.CompressTypeNoop), 462 client.WithTLS( 463 "../testdata/client.crt", 464 "../testdata/client.key", 465 "../testdata/ca.pem", 466 "localhost", 467 ), 468 )) 469 require.Equal(t, []byte(t.Name()), rsp.Data) 470 } 471 ``` 472 473 #### Client Certificate Not Authenticated 474 475 ##### Configuration Only 476 477 Simply add the corresponding configuration items (certificate and private key) in `trpc_go.yaml`: 478 479 ```yaml 480 server: # Server configuration 481 service: # Business services provided, can have multiple 482 - name: trpc.app.server.stdhttp 483 network: tcp 484 protocol: http_no_protocol # Fill in http for generic HTTP RPC services 485 tls_cert: "../testdata/server.crt" # Add certificate path 486 tls_key: "../testdata/server.key" # Add private key path 487 # ca_cert: "" # CA certificate, leave empty when the client certificate is not authenticated 488 client: # Client configuration 489 service: # Business services provided, can have multiple 490 - name: trpc.app.server.stdhttp 491 network: tcp 492 protocol: http 493 # tls_cert: "" # Certificate path, leave empty when the client certificate is not authenticated 494 # tls_key: "" # Private key path, leave empty when the client certificate is not authenticated 495 ca_cert: "none" # CA certificate, fill in "none" when the client certificate is not authenticated 496 ``` 497 498 For the mutual authentication part, the main difference is that the server's `ca_cert` needs to be left empty and the client's `ca_cert` needs to be filled with "none". 499 500 No additional TLS/HTTPS-related operations are needed in the code (no need to specify the scheme as `https`, no need to manually add the `WithTLS` option, and no need to find a way to include an HTTPS-related identifier in `WithTarget` or other places). 501 502 ##### Code Only 503 504 For the server, use `server.WithTLS` to specify the server certificate, private key, and leave the CA certificate empty: 505 506 ```go 507 server.WithTLS( 508 "../testdata/server.crt", 509 "../testdata/server.key", 510 "", // Leave the CA certificate empty when the client certificate is not authenticated 511 ) 512 ``` 513 514 For the client, use `client.WithTLS` to specify the client certificate, private key, and fill in "none" for the CA certificate: 515 516 ```go 517 client.WithTLS( 518 "", // Leave the certificate path empty 519 "", // Leave the private key path empty 520 "none", // Fill in "none" for the CA certificate when the client certificate is not authenticated 521 "", // Leave the server name empty 522 ) 523 ``` 524 525 No additional TLS/HTTPS-related operations are needed in the code. 526 527 Example: 528 529 ```go 530 func TestHTTPSSkipClientVerify(t *testing.T) { 531 const ( 532 network = "tcp" 533 address = "127.0.0.1:0" 534 ) 535 ln, err := net.Listen(network, address) 536 require.Nil(t, err) 537 defer ln.Close() 538 serviceName := "trpc.app.server.Service" + t.Name() 539 service := server.New( 540 server.WithServiceName(serviceName), 541 server.WithNetwork("tcp"), 542 server.WithProtocol("http_no_protocol"), 543 server.WithListener(ln), 544 server.WithTLS( 545 "../testdata/server.crt", 546 "../testdata/server.key", 547 "", 548 ), 549 ) 550 thttp.HandleFunc("/", func(w http.ResponseWriter, _ *http.Request) error { 551 w.Write([]byte(t.Name())) 552 return nil 553 }) 554 thttp.RegisterNoProtocolService(service) 555 s := &server.Server{} 556 s.AddService(serviceName, service) 557 go s.Serve() 558 defer s.Close(nil) 559 time.Sleep(100 * time.Millisecond) 560 561 c := thttp.NewClientProxy( 562 serviceName, 563 client.WithTarget("ip://"+ln.Addr().String()), 564 ) 565 req := &codec.Body{} 566 rsp := &codec.Body{} 567 require.Nil(t, 568 c.Post(context.Background(), "/", req, rsp, 569 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 570 client.WithSerializationType(codec.SerializationTypeNoop), 571 client.WithCurrentCompressType(codec.CompressTypeNoop), 572 client.WithTLS( 573 "", "", "none", "", 574 ), 575 )) 576 require.Equal(t, []byte(t.Name()), rsp.Data) 577 } 578 ``` 579 580 581 ### Client uses `io.Reader` for streaming file upload 582 583 Requires trpc-go version >= v0.13.0 584 585 The key point is to assign an `io.Reader` to the `thttp.ClientReqHeader.ReqBody` field (`body` is an `io.Reader`): 586 587 ```go 588 reqHeader := &thttp.ClientReqHeader{ 589 Header: header, 590 ReqBody: body, // Stream send. 591 } 592 ``` 593 594 Then specify `client.WithReqHead(reqHeader)` when making the call: 595 596 ```go 597 c.Post(context.Background(), "/", req, rsp, 598 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 599 client.WithSerializationType(codec.SerializationTypeNoop), 600 client.WithCurrentCompressType(codec.CompressTypeNoop), 601 client.WithReqHead(reqHeader), 602 ) 603 ``` 604 605 Here's an example: 606 607 ```go 608 func TestHTTPStreamFileUpload(t *testing.T) { 609 // Start server. 610 const ( 611 network = "tcp" 612 address = "127.0.0.1:0" 613 ) 614 ln, err := net.Listen(network, address) 615 require.Nil(t, err) 616 defer ln.Close() 617 go http.Serve(ln, &fileHandler{}) 618 // Start client. 619 c := thttp.NewClientProxy( 620 "trpc.app.server.Service_http", 621 client.WithTarget("ip://"+ln.Addr().String()), 622 ) 623 // Open and read file. 624 fileDir, err := os.Getwd() 625 require.Nil(t, err) 626 fileName := "README.md" 627 filePath := path.Join(fileDir, fileName) 628 file, err := os.Open(filePath) 629 require.Nil(t, err) 630 defer file.Close() 631 // Construct multipart form file. 632 body := &bytes.Buffer{} 633 writer := multipart.NewWriter(body) 634 part, err := writer.CreateFormFile("field_name", filepath.Base(file.Name())) 635 require.Nil(t, err) 636 io.Copy(part, file) 637 require.Nil(t, writer.Close()) 638 // Add multipart form data header. 639 header := http.Header{} 640 header.Add("Content-Type", writer.FormDataContentType()) 641 reqHeader := &thttp.ClientReqHeader{ 642 Header: header, 643 ReqBody: body, // Stream send. 644 } 645 req := &codec.Body{} 646 rsp := &codec.Body{} 647 // Upload file. 648 require.Nil(t, 649 c.Post(context.Background(), "/", req, rsp, 650 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 651 client.WithSerializationType(codec.SerializationTypeNoop), 652 client.WithCurrentCompressType(codec.CompressTypeNoop), 653 client.WithReqHead(reqHeader), 654 )) 655 require.Equal(t, []byte(fileName), rsp.Data) 656 } 657 658 type fileHandler struct{} 659 660 func (*fileHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { 661 _, h, err := r.FormFile("field_name") 662 if err != nil { 663 w.WriteHeader(http.StatusBadRequest) 664 return 665 } 666 w.WriteHeader(http.StatusOK) 667 // Write back file name. 668 w.Write([]byte(h.Filename)) 669 return 670 } 671 ``` 672 673 ### Reading Response Body Stream Using io.Reader in the Client 674 675 Requires trpc-go version >= v0.13.0 676 677 The key is to add `thttp.ClientRspHeader` and specify the `thttp.ClientRspHeader.ManualReadBody` field as `true`: 678 679 ```go 680 rspHead := &thttp.ClientRspHeader{ 681 ManualReadBody: true, 682 } 683 ``` 684 685 Then, when making the call, add `client.WithRspHead(rspHead)`: 686 687 ```go 688 c.Post(context.Background(), "/", req, rsp, 689 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 690 client.WithSerializationType(codec.SerializationTypeNoop), 691 client.WithCurrentCompressType(codec.CompressTypeNoop), 692 client.WithRspHead(rspHead), 693 ) 694 ``` 695 696 Finally, you can perform streaming reads on `rspHead.Response.Body`: 697 698 ```go 699 body := rspHead.Response.Body // Do stream reads directly from rspHead.Response.Body. 700 defer body.Close() // Do remember to close the body. 701 bs, err := io.ReadAll(body) 702 ``` 703 704 Here's an example: 705 706 ```go 707 func TestHTTPStreamRead(t *testing.T) { 708 // Start server. 709 const ( 710 network = "tcp" 711 address = "127.0.0.1:0" 712 ) 713 ln, err := net.Listen(network, address) 714 require.Nil(t, err) 715 defer ln.Close() 716 go http.Serve(ln, &fileServer{}) 717 718 // Start client. 719 c := thttp.NewClientProxy( 720 "trpc.app.server.Service_http", 721 client.WithTarget("ip://"+ln.Addr().String()), 722 ) 723 724 // Enable manual body reading in order to 725 // disable the framework's automatic body reading capability, 726 // so that users can manually do their own client-side streaming reads. 727 rspHead := &thttp.ClientRspHeader{ 728 ManualReadBody: true, 729 } 730 req := &codec.Body{} 731 rsp := &codec.Body{} 732 require.Nil(t, 733 c.Post(context.Background(), "/", req, rsp, 734 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 735 client.WithSerializationType(codec.SerializationTypeNoop), 736 client.WithCurrentCompressType(codec.CompressTypeNoop), 737 client.WithRspHead(rspHead), 738 )) 739 require.Nil(t, rsp.Data) 740 body := rspHead.Response.Body // Do stream reads directly from rspHead.Response.Body. 741 defer body.Close() // Do remember to close the body. 742 bs, err := io.ReadAll(body) 743 require.Nil(t, err) 744 require.NotNil(t, bs) 745 } 746 747 type fileServer struct{} 748 749 func (*fileServer) ServeHTTP(w http.ResponseWriter, r *http.Request) { 750 http.ServeFile(w, r, "./README.md") 751 return 752 } 753 ``` 754 755 ### Client and Server Sending and Receiving HTTP Chunked 756 757 1. Client sends HTTP chunked: 758 1. Add `chunked` Transfer-Encoding header. 759 2. Use io.Reader to send the data. 760 2. Client receives HTTP chunked: The Go standard library's HTTP automatically supports handling chunked responses. The upper-level user is unaware of it and only needs to loop over reading from `resp.Body` until `io.EOF` (or use `io.ReadAll`). 761 3. Server reads HTTP chunked: Similar to client reading. 762 4. Server sends HTTP chunked: Assert `http.ResponseWriter` as `http.Flusher`, then call `flusher.Flush()` after sending a portion of the data. This will automatically trigger the `chunked` encoding and send a chunk. 763 764 Here is an example: 765 766 ```go 767 func TestHTTPSendReceiveChunk(t *testing.T) { 768 // HTTP chunked example: 769 // 1. Client sends chunks: Add "chunked" transfer encoding header, and use io.Reader as body. 770 // 2. Client reads chunks: The Go/net/http automatically handles the chunked reading. 771 // Users can simply read resp.Body in a loop until io.EOF. 772 // 3. Server reads chunks: Similar to client reads chunks. 773 // 4. Server sends chunks: Assert http.ResponseWriter as http.Flusher, call flusher.Flush() after 774 // writing a part of data, it will automatically trigger "chunked" encoding to send a chunk. 775 776 // Start server. 777 const ( 778 network = "tcp" 779 address = "127.0.0.1:0" 780 ) 781 ln, err := net.Listen(network, address) 782 require.Nil(t, err) 783 defer ln.Close() 784 go http.Serve(ln, &chunkedServer{}) 785 786 // Start client. 787 c := thttp.NewClientProxy( 788 "trpc.app.server.Service_http", 789 client.WithTarget("ip://"+ln.Addr().String()), 790 ) 791 792 // Open and read file. 793 fileDir, err := os.Getwd() 794 require.Nil(t, err) 795 fileName := "README.md" 796 filePath := path.Join(fileDir, fileName) 797 file, err := os.Open(filePath) 798 require.Nil(t, err) 799 defer file.Close() 800 801 // 1. Client sends chunks. 802 803 // Add request headers. 804 header := http.Header{} 805 header.Add("Content-Type", "text/plain") 806 // Add chunked transfer encoding header. 807 header.Add("Transfer-Encoding", "chunked") 808 reqHead := &thttp.ClientReqHeader{ 809 Header: header, 810 ReqBody: file, // Stream send (for chunks). 811 } 812 813 // Enable manual body reading in order to 814 // disable the framework's automatic body reading capability, 815 // so that users can manually do their own client-side streaming reads. 816 rspHead := &thttp.ClientRspHeader{ 817 ManualReadBody: true, 818 } 819 req := &codec.Body{} 820 rsp := &codec.Body{} 821 require.Nil(t, 822 c.Post(context.Background(), "/", req, rsp, 823 client.WithCurrentSerializationType(codec.SerializationTypeNoop), 824 client.WithSerializationType(codec.SerializationTypeNoop), 825 client.WithCurrentCompressType(codec.CompressTypeNoop), 826 client.WithReqHead(reqHead), 827 client.WithRspHead(rspHead), 828 )) 829 require.Nil(t, rsp.Data) 830 831 // 2. Client reads chunks. 832 833 // Do stream reads directly from rspHead.Response.Body. 834 body := rspHead.Response.Body 835 defer body.Close() // Do remember to close the body. 836 buf := make([]byte, 4096) 837 var idx int 838 for { 839 n, err := body.Read(buf) 840 if err == io.EOF { 841 t.Logf("reached io.EOF\n") 842 break 843 } 844 t.Logf("read chunk %d of length %d: %q\n", idx, n, buf[:n]) 845 idx++ 846 } 847 } 848 849 type chunkedServer struct{} 850 851 func (*chunkedServer) ServeHTTP(w http.ResponseWriter, r *http.Request) { 852 // 3. Server reads chunks. 853 854 // io.ReadAll will read until io.EOF. 855 // Go/net/http will automatically handle chunked body reads. 856 bs, err := io.ReadAll(r.Body) 857 if err != nil { 858 w.WriteHeader(http.StatusInternalServerError) 859 w.Write([]byte(fmt.Sprintf("io.ReadAll err: %+v", err))) 860 return 861 } 862 863 // 4. Server sends chunks. 864 865 // Send HTTP chunks using http.Flusher. 866 // Reference: https://stackoverflow.com/questions/26769626/send-a-chunked-http-response-from-a-go-server. 867 // The "Transfer-Encoding" header will be handled by the writer implicitly, so no need to set it. 868 flusher, ok := w.(http.Flusher) 869 if !ok { 870 w.WriteHeader(http.StatusInternalServerError) 871 w.Write([]byte("expected http.ResponseWriter to be an http.Flusher")) 872 return 873 } 874 chunks := 10 875 chunkSize := (len(bs) + chunks - 1) / chunks 876 for i := 0; i < chunks; i++ { 877 start := i * chunkSize 878 end := (i + 1) * chunkSize 879 if end > len(bs) { 880 end = len(bs) 881 } 882 w.Write(bs[start:end]) 883 flusher.Flush() // Trigger "chunked" encoding and send a chunk. 884 time.Sleep(500 * time.Millisecond) 885 } 886 return 887 } 888 ```