github.com/thanos-io/thanos@v0.32.5/docs/contributing/coding-style-guide.md (about) 1 # Thanos Coding Style Guide 2 3 This document details the official style guides for the various languages we use in the Thanos project. Feel free to familiarize yourself with and refer to this document during code reviews. If something in our codebase does not match the style, it means it was missed or it was written before this document. Help wanted to fix it! (: 4 5 Generally, we care about: 6 7 * Readability, so low [Cognitive Load](https://www.dabapps.com/blog/cognitive-load-programming/). 8 * Maintainability. We avoid the code that **surprises**. 9 * Performance only for critical path and without compromising readability. 10 * Testability. Even if it means some changes to the production code, like `timeNow func() time.Time` mock. 11 * Consistency: If some pattern repeats, it means fewer surprises. 12 13 Some style is enforced by our linters and is covered in separate smaller sections. Please look there if you want to embrace some of the rules in your own project! For Thanos developers, we recommend reading sections about rules to manually apply during development. Some of those are currently impossible to detect with linters. Ideally, everything would be automated. (: 14 15 # Go 16 17 For code written in [Go](https://golang.org/) we use the standard Go style guides ([Effective Go](https://golang.org/doc/effective_go.html), [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments)) with a few additional rules that make certain areas stricter than the standard guides. This ensures even better consistency in modern distributed system databases like Thanos, where reliability, performance, and maintainability are extremely important. 18 19 <img src="../img/go-in-thanos.jpg" class="img-fluid" alt="Go in Thanos"/> 20 21 <!-- 22 NOTE: Because of blackfriday bug, we have to change those code snippet to `< highlight go >` hugo shortcodes during `websitepreprocessing.sh` for Thanos website. 23 --> 24 25 ## Development / Code Review 26 27 In this section, we will go through rules that on top of the standard guides that we apply during development and code reviews. 28 29 NOTE: If you know that any of those rules can be enabled by some linter, automatically, let us know! (: 30 31 ### Reliability 32 33 The coding style is not purely about what is ugly and what is not. It's mainly to make sure programs are reliable for running on production 24h per day without causing incidents. The following rules are describing some unhealthy patterns we have seen across the Go community that are often forgotten. Those things can be considered bugs or can significantly increase the chances of introducing a bug. 34 35 #### Defers: Don't Forget to Check Returned Errors 36 37 It's easy to forget to check the error returned by a `Close` method that we deferred. 38 39 ```go 40 f, err := os.Open(...) 41 if err != nil { 42 // handle.. 43 } 44 defer f.Close() // What if an error occurs here? 45 46 // Write something to file... etc. 47 ``` 48 49 Unchecked errors like this can lead to major bugs. Consider the above example: the `*os.File` `Close` method can be responsible for actually flushing to the file, so if an error occurs at that point, the whole **write might be aborted!** 😱 50 51 Always check errors! To make it consistent and not distracting, use our [runutil](https://pkg.go.dev/github.com/thanos-io/thanos@v0.17.0/pkg/runutil?tab=doc) helper package, e.g.: 52 53 ```go 54 // Use `CloseWithErrCapture` if you want to close and fail the function or 55 // method on a `f.Close` error (make sure the `error` return argument is 56 // named as `err`). If the error is already present, `CloseWithErrCapture` 57 // will append (not wrap) the `f.Close` error if any. 58 defer runutil.CloseWithErrCapture(&err, f, "close file") 59 60 // Use `CloseWithLogOnErr` if you want to close and log error on `Warn` 61 // level on a `f.Close` error. 62 defer runutil.CloseWithLogOnErr(logger, f, "close file") 63 ``` 64 65 <table> 66 <tbody> 67 <tr><th>Avoid 🔥</th></tr> 68 <tr><td> 69 70 ```go 71 func writeToFile(...) error { 72 f, err := os.Open(...) 73 if err != nil { 74 return err 75 } 76 defer f.Close() // What if an error occurs here? 77 78 // Write something to file... 79 return nil 80 } 81 ``` 82 83 </td></tr> 84 <tr><th>Better 🤓</th></tr> 85 <tr><td> 86 87 ```go 88 func writeToFile(...) (err error) { 89 f, err := os.Open(...) 90 if err != nil { 91 return err 92 } 93 // Now all is handled well. 94 defer runutil.CloseWithErrCapture(&err, f, "close file") 95 96 // Write something to file... 97 return nil 98 } 99 ``` 100 101 </td></tr> 102 </tbody></table> 103 104 #### Exhaust Readers 105 106 One of the most common bugs is forgetting to close or fully read the bodies of HTTP requests and responses, especially on error. If you read the body of such structures, you can use the [runutil](https://pkg.go.dev/github.com/thanos-io/thanos@v0.17.0/pkg/runutil?tab=doc) helper as well: 107 108 ```go 109 defer runutil.ExhaustCloseWithLogOnErr(logger, resp.Body, "close response") 110 ``` 111 112 <table> 113 <tbody> 114 <tr><th>Avoid 🔥</th></tr> 115 <tr><td> 116 117 ```go 118 resp, err := http.Get("http://example.com/") 119 if err != nil { 120 // handle... 121 } 122 defer runutil.CloseWithLogOnErr(logger, resp.Body, "close response") 123 124 scanner := bufio.NewScanner(resp.Body) 125 // If any error happens and we return in the middle of scanning 126 // body, we can end up with unread buffer, which 127 // will use memory and hold TCP connection! 128 for scanner.Scan() { 129 ``` 130 131 </td></tr> 132 <tr><th>Better 🤓</th></tr> 133 <tr><td> 134 135 ```go 136 resp, err := http.Get("http://example.com/") 137 if err != nil { 138 // handle... 139 } 140 defer runutil.ExhaustCloseWithLogOnErr(logger, resp.Body, "close response") 141 142 scanner := bufio.NewScanner(resp.Body) 143 // If any error happens and we return in the middle of scanning body, 144 // defer will handle all well. 145 for scanner.Scan() { 146 ``` 147 148 </td></tr> 149 </tbody></table> 150 151 #### Avoid Globals 152 153 No globals other than `const` are allowed. Period. This means also, no `init` functions. 154 155 #### Never Use Panics 156 157 Never use them. If some dependency uses it, use [recover](https://golang.org/doc/effective_go.html#recover). Also, consider avoiding that dependency. 🙈 158 159 #### Avoid Using the `reflect` or `unsafe` Packages 160 161 Use those only for very specific, critical cases. Especially `reflect` tend to be very slow. For testing code, it's fine to use reflect. 162 163 #### Avoid variable shadowing 164 165 Variable shadowing is when you use the same variable name in a smaller scope that "shadows". This is very dangerous as it leads to many surprises. It's extremely hard to debug such problems as they might appear in unrelated parts of the code. And what's broken is tiny `:` or lack of it. 166 167 <table> 168 <tbody> 169 <tr><th>Avoid 🔥</th></tr> 170 <tr><td> 171 172 ```go 173 var client ClientInterface 174 if clientTypeASpecified { 175 // Ups - typo, should be =` 176 client, err := clienttypea.NewClient(...) 177 if err != nil { 178 // handle err 179 } 180 level.Info(logger).Log("msg", "created client", "type", client.Type) 181 } else { 182 // Ups - typo, should be =` 183 client, err := clienttypea.NewClient(...) 184 level.Info(logger).Log("msg", "noop client will be used", "type", client.Type) 185 } 186 187 // In some further deeper part of the code... 188 resp, err := client.Call(....) // nil pointer panic! 189 ``` 190 191 </td></tr> 192 <tr><th>Better 🤓</th></tr> 193 <tr><td> 194 195 ```go 196 var client ClientInterface = NewNoop(...) 197 if clientTypeASpecified { 198 c, err := clienttypea.NewClient(...) 199 if err != nil { 200 // handle err 201 } 202 client = c 203 } 204 level.Info(logger).Log("msg", "created client", "type", c.Type) 205 206 resp, err := client.Call(....) 207 ``` 208 209 </td></tr> 210 </tbody></table> 211 212 This is also why we recommend to scope errors if you can: 213 214 ```go 215 if err := doSomething(); err != nil { 216 // handle err 217 } 218 ``` 219 220 While it's not yet configured, we might think consider not permitting variable shadowing with [`golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow`](https://godoc.org/golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow) in future. There was even Go 2 proposal for [disabling this in the language itself, but was rejected](https://github.com/golang/go/issues/21114): 221 222 Similar to this problem is the package name shadowing. While it is less dangerous, it can cause similar issues, so avoid package shadowing if you can. 223 224 ### Performance 225 226 After all, Thanos system is a database that has to perform queries over terabytes of data within human friendly response times. This might require some additional patterns in our code. With those patterns, we try to not sacrifice the readability and apply those only on the critical code paths. 227 228 **Keep in mind to always measure the results.** The Go performance relies on many hidden things and tweaks, so the good micro benchmark, following with the real system load test is in most times required to tell if optimization makes sense. 229 230 #### Pre-allocating Slices and Maps 231 232 Try to always preallocate slices and map. If you know the number of elements you want to put apriori, use that knowledge! This significantly improves the latency of such code. Consider this as micro optimization, however, it's a good pattern to do it always, as it does not add much complexity. Performance wise, it's only relevant for critical, code paths with big arrays. 233 234 NOTE: This is because, in very simple view, the Go runtime allocates 2 times the current size. So if you expect million of elements, Go will do many allocations on `append` in between instead of just one if you preallocate. 235 236 <table> 237 <tbody> 238 <tr><th>Avoid 🔥</th></tr> 239 <tr><td> 240 241 ```go 242 func copyIntoSliceAndMap(biggy []string) (a []string, b map[string]struct{}) 243 b = map[string]struct{}{} 244 245 for _, item := range biggy { 246 a = append(a, item) 247 b[item] = struct{} 248 } 249 } 250 ``` 251 252 </td></tr> 253 <tr><th>Better 🤓</th></tr> 254 <tr><td> 255 256 ```go 257 func copyIntoSliceAndMap(biggy []string) (a []string, b map[string]struct{}) 258 b = make(map[string]struct{}, len(biggy)) 259 a = make([]string, len(biggy)) 260 261 // Copy will not even work without pre-allocation. 262 copy(a, biggy) 263 for _, item := range biggy { 264 b[item] = struct{} 265 } 266 } 267 ``` 268 269 </td></tr> 270 </tbody></table> 271 272 #### Reuse arrays 273 274 To extend the above point, there are cases where you don't need to allocate new space in memory all the time. If you repeat the certain operation on slices sequentially and you just release the array on every iteration, it's reasonable to reuse the underlying array for those. This can give quite enormous gains for critical paths. Unfortunately, currently there is no way to reuse the underlying array for maps. 275 276 NOTE: Why you cannot just allocate slice and release and in new iteration allocate and release again etc? Go should know it has available space and just reuses that no? (: Well, it's not that easy. TL;DR is that Go Garbage Collection runs periodically or on certain cases (big heap), but definitely not on every iteration of your loop (that would be super slow). Read more in details [here](http://web.archive.org/web/20220511045405/https://about.sourcegraph.com/go/gophercon-2018-allocator-wrestling/). 277 278 <table> 279 <tbody> 280 <tr><th>Avoid 🔥</th></tr> 281 <tr><td> 282 283 ```go 284 var messages []string 285 for _, msg := range recv { 286 messages = append(messages, msg) 287 288 if len(messages) > maxMessageLen { 289 marshalAndSend(messages) 290 // This creates new array. Previous array 291 // will be garbage collected only after 292 // some time (seconds), which 293 // can create enormous memory pressure. 294 messages = []string 295 } 296 } 297 ``` 298 299 </td></tr> 300 <tr><th>Better 🤓</th></tr> 301 <tr><td> 302 303 ```go 304 var messages []string 305 for _, msg := range recv { 306 messages = append(messages, msg) 307 308 if len(messages) > maxMessageLen { 309 marshalAndSend(messages) 310 // Instead of new array, reuse 311 // the same, with the same capacity, 312 // just length equals to zero. 313 messages = messages[:0] 314 } 315 } 316 ``` 317 318 </td></tr> 319 </tbody></table> 320 321 ### Readability 322 323 The part that all Gophers love ❤️ How to make code more readable? 324 325 For the Thanos Team, readability is about programming in a way that does not surprise the reader of the code. All the details and inconsistencies can distract or mislead the reader, so every character or newline might matter. That's why we might be spending more time on every Pull Requests' review, especially in the beginning, but for a good reason! To make sure we can quickly understand, extend and fix problems with our system. 326 327 #### Keep the Interface Narrow; Avoid Shallow Functions 328 329 This is connected more to the API design than coding, but even during small coding decisions it matter. For example how you define functions or methods. There are two general rules: 330 331 * Simpler (usually it means smaller) interfaces are better. This might mean a smaller, simpler function signature as well as fewer methods in the interfaces. Try to group interfaces based on functionality to expose at max 1-3 methods if possible. 332 333 <table> 334 <tbody> 335 <tr><th>Avoid 🔥</th></tr> 336 <tr><td> 337 338 ```go 339 // Compactor aka: The Big Boy. Such big interface is really useless ): 340 type Compactor interface { 341 Compact(ctx context.Context) error 342 FetchMeta(ctx context.Context) (metas map[ulid.ULID]*metadata.Meta, partial map[ulid.ULID]error, err error) 343 UpdateOnMetaChange(func([]metadata.Meta, error)) 344 SyncMetas(ctx context.Context) error 345 Groups() (res []*Group, err error) 346 GarbageCollect(ctx context.Context) error 347 ApplyRetentionPolicyByResolution(ctx context.Context, logger log.Logger, bkt objstore.Bucket) error 348 BestEffortCleanAbortedPartialUploads(ctx context.Context, bkt objstore.Bucket) 349 DeleteMarkedBlocks(ctx context.Context) error 350 Downsample(ctx context.Context, logger log.Logger, metrics *DownsampleMetrics, bkt objstore.Bucket) error 351 } 352 ``` 353 354 </td></tr> 355 <tr><th>Better 🤓</th></tr> 356 <tr><td> 357 358 ```go 359 // Smaller interfaces with a smaller number of arguments allow functional grouping, clean composition and clear testability. 360 type Compactor interface { 361 Compact(ctx context.Context) error 362 } 363 364 type Downsampler interface { 365 Downsample(ctx context.Context) error 366 } 367 368 type MetaFetcher interface { 369 Fetch(ctx context.Context) (metas map[ulid.ULID]*metadata.Meta, partial map[ulid.ULID]error, err error) 370 UpdateOnChange(func([]metadata.Meta, error)) 371 } 372 373 type Syncer interface { 374 SyncMetas(ctx context.Context) error 375 Groups() (res []*Group, err error) 376 GarbageCollect(ctx context.Context) error 377 } 378 379 type RetentionKeeper interface { 380 Apply(ctx context.Context) error 381 } 382 383 type Cleaner interface { 384 DeleteMarkedBlocks(ctx context.Context) error 385 BestEffortCleanAbortedPartialUploads(ctx context.Context) 386 } 387 ``` 388 389 </td></tr> 390 </tbody></table> 391 392 * It's better if you can hide more unnecessary complexity from the user. This means that having shallow function introduce more cognitive load to understand the function name or navigate to implementation to understand it better. It might be much more readable to inline those few lines directly on the caller side. 393 394 <table> 395 <tbody> 396 <tr><th>Avoid 🔥</th></tr> 397 <tr><td> 398 399 ```go 400 // Some code... 401 s.doSomethingAndHandleError() 402 403 // Some code... 404 } 405 406 func (s *myStruct) doSomethingAndHandleError() { 407 if err := doSomething(); err != nil { 408 level.Error(s.logger).Log("msg" "failed to do something; sorry", "err", err) 409 } 410 } 411 ``` 412 413 </td></tr> 414 <tr><th>Better 🤓</th></tr> 415 <tr><td> 416 417 ```go 418 // Some code... 419 if err := doSomething(); err != nil { 420 level.Error(s.logger).Log("msg" "failed to do something; sorry", "err", err) 421 } 422 423 // Some code... 424 } 425 ``` 426 427 </td></tr> 428 </tbody></table> 429 430 This is a little bit connected to `There should be one-- and preferably only one --obvious way to do it` and [`DRY`](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) rules. If you have more ways of doing something than one, it means you have a wider interface, allowing more opportunities for errors, ambiguity and maintenance burden. 431 432 <table> 433 <tbody> 434 <tr><th>Avoid 🔥</th></tr> 435 <tr><td> 436 437 ```go 438 // We have here SIX potential ways the caller can get an ID. Can you find all of them? 439 440 type Block struct { 441 // Things... 442 ID ulid.ULID 443 444 mtx sync.Mutex 445 } 446 447 func (b *Block) Lock() { b.mtx.Lock() } 448 449 func (b *Block) Unlock() { b.mtx.Unlock() } 450 451 func (b *Block) ID() ulid.ULID { 452 b.mtx.Lock() 453 defer b.mtx.Unlock() 454 return b.ID 455 } 456 457 func (b *Block) IDNoLock() ulid.ULID { return b.ID } 458 ``` 459 460 </td></tr> 461 <tr><th>Better 🤓</th></tr> 462 <tr><td> 463 464 ```go 465 type Block struct { 466 // Things... 467 468 id ulid.ULID 469 mtx sync.Mutex 470 } 471 472 func (b *Block) ID() ulid.ULID { 473 b.mtx.Lock() 474 defer b.mtx.Unlock() 475 return b.id 476 } 477 ``` 478 479 </td></tr> 480 </tbody></table> 481 482 #### Use Named Return Parameters Carefully 483 484 It's OK to name return parameters if the types do not give enough information about what function or method actually returns. Another use case is when you want to define a variable, e.g. a slice. 485 486 **IMPORTANT:** never use naked `return` statements with named return parameters. This compiles but it makes returning values implicit and thus more prone to surprises. 487 488 #### Clean Defer Only if Function Fails 489 490 There is a way to sacrifice defer in order to properly close all on each error. Repetition makes it easier to make an error and forget something when changing the code, so on-error deferring is doable: 491 492 <table> 493 <tbody> 494 <tr><th>Avoid 🔥</th></tr> 495 <tr><td> 496 497 ```go 498 func OpenSomeFileAndDoSomeStuff() (*os.File, error) { 499 f, err := os.OpenFile("file.txt", os.O_RDONLY, 0) 500 if err != nil { 501 return nil, err 502 } 503 504 if err := doStuff1(); err != nil { 505 runutil.CloseWithErrCapture(&err, f, "close file") 506 return nil, err 507 } 508 if err := doStuff2(); err != nil { 509 runutil.CloseWithErrCapture(&err, f, "close file") 510 return nil, err 511 } 512 if err := doStuff232241(); err != nil { 513 // Ups.. forgot to close file here. 514 return nil, err 515 } 516 return f, nil 517 } 518 ``` 519 520 </td></tr> 521 <tr><th>Better 🤓</th></tr> 522 <tr><td> 523 524 ```go 525 func OpenSomeFileAndDoSomeStuff() (f *os.File, err error) { 526 f, err = os.OpenFile("file.txt", os.O_RDONLY, 0) 527 if err != nil { 528 return nil, err 529 } 530 defer func() { 531 if err != nil { 532 runutil.CloseWithErrCapture(&err, f, "close file") 533 } 534 }() 535 536 if err := doStuff1(); err != nil { 537 return nil, err 538 } 539 if err := doStuff2(); err != nil { 540 return nil, err 541 } 542 if err := doStuff232241(); err != nil { 543 return nil, err 544 } 545 return f, nil 546 } 547 ``` 548 549 </td></tr> 550 </tbody></table> 551 552 #### Explicitly Handle Returned Errors 553 554 Always handle returned errors. It does not mean you cannot "ignore" the error for some reason, e.g. if we know implementation will not return anything meaningful. You can ignore the error, but do so explicitly: 555 556 <table> 557 <tbody> 558 <tr><th>Avoid 🔥</th></tr> 559 <tr><td> 560 561 ```go 562 someMethodThatReturnsError(...) 563 ``` 564 565 </td></tr> 566 <tr><th>Better 🤓</th></tr> 567 <tr><td> 568 569 ```go 570 _ = someMethodThatReturnsError(...) 571 ``` 572 573 </td></tr> 574 </tbody></table> 575 576 The exception: well-known cases such as `level.Debug|Warn` etc and `fmt.Fprint*` 577 578 #### Avoid Defining Variables Used Only Once. 579 580 It's tempting to define a variable as an intermittent step to create something bigger. Avoid defining such a variable if it's used only once. When you create a variable *the reader* expects some other usage of this variable than one, so it can be annoying to every time double check that and realize that it's only used once. 581 582 <table> 583 <tbody> 584 <tr><th>Avoid 🔥</th></tr> 585 <tr><td> 586 587 ```go 588 someConfig := a.GetConfig() 589 address124 := someConfig.Addresses[124] 590 addressStr := fmt.Sprintf("%s:%d", address124.Host, address124.Port) 591 592 c := &MyType{HostPort: addressStr, SomeOther: thing} 593 return c 594 ``` 595 596 </td></tr> 597 <tr><th>Better 🤓</th></tr> 598 <tr><td> 599 600 ```go 601 // This variable is required for potentially consistent results. It is used twice. 602 someConfig := a.FetchConfig() 603 return &MyType{ 604 HostPort: fmt.Sprintf("%s:%d", someConfig.Addresses[124].Host, someConfig.Addresses[124].Port), 605 SomeOther: thing, 606 } 607 ``` 608 609 </td></tr> 610 </tbody></table> 611 612 #### Only Two Ways of Formatting Functions/Methods 613 614 Prefer function/method definitions with arguments in a single line. If it's too wide, put each argument on a new line. 615 616 <table> 617 <tbody> 618 <tr><th>Avoid 🔥</th></tr> 619 <tr><td> 620 621 ```go 622 func function(argument1 int, argument2 string, 623 argument3 time.Duration, argument4 someType, 624 argument5 float64, argument6 time.Time, 625 ) (ret int, err error) { 626 ``` 627 628 </td></tr> 629 <tr><th>Better 🤓</th></tr> 630 <tr><td> 631 632 ```go 633 func function( 634 argument1 int, 635 argument2 string, 636 argument3 time.Duration, 637 argument4 someType, 638 argument5 float64, 639 argument6 time.Time, 640 ) (ret int, err error) 641 ``` 642 643 </td></tr> 644 </tbody></table> 645 646 This applies for both calling and defining method / function. 647 648 NOTE: One exception would be when you expect the variadic (e.g. `...string`) arguments to be filled in pairs, e.g: 649 650 ```go 651 level.Info(logger).Log( 652 "msg", "found something epic during compaction; this looks amazing", 653 "compNumber", compNumber, 654 "block", id, 655 "elapsed", timeElapsed, 656 ) 657 ``` 658 659 #### Control Structure: Prefer early returns and avoid `else` 660 661 In most of the cases, you don't need `else`. You can usually use `continue`, `break` or `return` to end an `if` block. This enables having one less indent and better consistency so code is more readable. 662 663 <table> 664 <tbody> 665 <tr><th>Avoid 🔥</th></tr> 666 <tr><td> 667 668 ```go 669 for _, elem := range elems { 670 if a == 1 { 671 something[i] = "yes" 672 } else 673 something[i] = "no" 674 } 675 } 676 ``` 677 678 </td></tr> 679 <tr><th>Better 🤓</th></tr> 680 <tr><td> 681 682 ```go 683 for _, elem := range elems { 684 if a == 1 { 685 something[i] = "yes" 686 continue 687 } 688 something[i] = "no" 689 } 690 ``` 691 692 </td></tr> 693 </tbody></table> 694 695 #### Wrap Errors for More Context; Don't Repeat "failed ..." There. 696 697 We use [`pkg/errors`](https://github.com/pkg/errors) package for `errors`. We prefer it over standard wrapping with `fmt.Errorf` + `%w`, as `errors.Wrap` is explicit. It's easy to by accident replace `%w` with `%v` or to add extra inconsistent characters to the string. 698 699 Use [`pkg/errors.Wrap`](https://github.com/pkg/errors) to wrap errors for future context when errors occur. It's recommended to add more interesting variables to add context using `errors.Wrapf`, e.g. file names, IDs or things that fail, etc. 700 701 NOTE: never prefix wrap messages with wording like `failed ... ` or `error occurred while...`. Just describe what we wanted to do when the failure occurred. Those prefixes are just noise. We are wrapping error, so it's obvious that some error occurred, right? (: Improve readability and consider avoiding those. 702 703 <table> 704 <tbody> 705 <tr><th>Avoid 🔥</th></tr> 706 <tr><td> 707 708 ```go 709 if err != nil { 710 return fmt.Errorf("error while reading from file %s: %w", f.Name, err) 711 } 712 ``` 713 714 </td></tr> 715 <tr><th>Better 🤓</th></tr> 716 <tr><td> 717 718 ```go 719 if err != nil { 720 return errors.Wrapf(err, "read file %s", f.Name) 721 } 722 ``` 723 724 </td></tr> 725 </tbody></table> 726 727 #### Use the Blank Identifier `_` 728 729 Blank identifiers are very useful to mark variables that are not used. Consider the following cases: 730 731 ```go 732 // We don't need the second return parameter. 733 // Let's use the blank identifier instead. 734 a, _, err := function1(...) 735 if err != nil { 736 // handle err 737 } 738 ``` 739 740 ```go 741 // We don't need to use this variable, we 742 // just want to make sure TypeA implements InterfaceA. 743 var _ InterfaceA = TypeA 744 ``` 745 746 ```go 747 // We don't use context argument; let's use the blank 748 // identifier to make it clear. 749 func (t *Type) SomeMethod(_ context.Context, abc int) error { 750 ``` 751 752 #### Rules for Log Messages 753 754 We use [go-kit logger](https://github.com/go-kit/kit/tree/master/log) in Thanos. This means that we expect log lines to have a certain structure. Structure means that instead of adding variables to the message, those should be passed as separate fields. Keep in mind that all log lines in Thanos should be `lowercase` (readability and consistency) and all struct keys are using `camelCase`. It's suggested to keep key names short and consistent. For example, if we always use `block` for block ID, let's not use in the other single log message `id`. 755 756 <table> 757 <tbody> 758 <tr><th>Avoid 🔥</th></tr> 759 <tr><td> 760 761 ```go 762 level.Info(logger).Log("msg", fmt.Sprintf("Found something epic during compaction number %v. This looks amazing.", compactionNumber), 763 "block_id", id, "elapsed-time", timeElapsed) 764 ``` 765 766 </td></tr> 767 <tr><th>Better 🤓</th></tr> 768 <tr><td> 769 770 ```go 771 level.Info(logger).Log("msg", "found something epic during compaction; this looks amazing", "compNumber", compNumber, 772 "block", id, "elapsed", timeElapsed) 773 ``` 774 775 </td></tr> 776 </tbody></table> 777 778 Additionally, there are certain rules we suggest while using different log levels: 779 780 * level.Info: Should always have `msg` field. It should be used only for important events that we expect to happen not too often. 781 * level.Debug: Should always have `msg` field. It can be a bit more spammy, but should not be everywhere as well. Use it only when you want to really dive into some problems in certain areas. 782 * level.Warn: Should have either `msg` or `err` or both fields. They should warn about events that are suspicious and to investigate but the process can gracefully mitigate it. Always try to describe *how* it was mitigated, what action will be performed e.g. `value will be skipped` 783 * level.Error: Should have either `msg` or `err` or both fields. Use it only for a critical event. 784 785 #### Comment Necessary Surprises 786 787 Comments are not the best. They age quickly and the compiler does not fail if you will forget to update them. So use comments only when necessary. **And it is necessary to comment on code that can surprise the user.** Sometimes, complexity is necessary, for example for performance. Comment in this case why such optimization was needed. If something was done temporarily add `TODO(<github name>): <something, with GitHub issue link ideally>`. 788 789 ### Testing 790 791 #### Table Tests 792 793 Use table-driven tests that use [t.Run](https://blog.golang.org/subtests) for readability. They are easy to read and allows to add a clean description of each test case. Adding or adapting test cases is also easier. 794 795 <table> 796 <tbody> 797 <tr><th>Avoid 🔥</th></tr> 798 <tr><td> 799 800 ```go 801 host, port, err := net.SplitHostPort("1.2.3.4:1234") 802 testutil.Ok(t, err) 803 testutil.Equals(t, "1.2.3.4", host) 804 testutil.Equals(t, "1234", port) 805 806 host, port, err = net.SplitHostPort("1.2.3.4:something") 807 testutil.Ok(t, err) 808 testutil.Equals(t, "1.2.3.4", host) 809 testutil.Equals(t, "http", port) 810 811 host, port, err = net.SplitHostPort(":1234") 812 testutil.Ok(t, err) 813 testutil.Equals(t, "", host) 814 testutil.Equals(t, "1234", port) 815 816 host, port, err = net.SplitHostPort("yolo") 817 testutil.NotOk(t, err) 818 ``` 819 820 </td></tr> 821 <tr><th>Better 🤓</th></tr> 822 <tr><td> 823 824 ```go 825 for _, tcase := range []struct{ 826 name string 827 828 input string 829 830 expectedHost string 831 expectedPort string 832 expectedErr error 833 }{ 834 { 835 name: "host and port", 836 837 input: "1.2.3.4:1234", 838 expectedHost: "1.2.3.4", 839 expectedPort: "1234", 840 }, 841 { 842 name: "host and named port", 843 844 input: "1.2.3.4:something", 845 expectedHost: "1.2.3.4", 846 expectedPort: "something", 847 }, 848 { 849 name: "just port", 850 851 input: ":1234", 852 expectedHost: "", 853 expectedPort: "1234", 854 }, 855 { 856 name: "not valid hostport", 857 858 input: "yolo", 859 expectedErr: errors.New("<exact error>") 860 }, 861 }{ 862 t.Run(tcase.name, func(t *testing.T) { 863 host, port, err := net.SplitHostPort(tcase.input) 864 if tcase.expectedErr != nil { 865 testutil.NotOk(t, err) 866 testutil.Equals(t, tcase.expectedErr, err) 867 return 868 } 869 testutil.Ok(t, err) 870 testutil.Equals(t, tcase.expectedHost, host) 871 testutil.Equals(t, tcase.expectedPort, port) 872 }) 873 } 874 ``` 875 876 </td></tr> 877 </tbody></table> 878 879 #### Tests for Packages / Structs That Involve `time` package. 880 881 Avoid unit testing based on real-time. Always try to mock time that is used within struct by using, for example, a `timeNow func() time.Time` field. For production code, you can initialize the field with `time.Now`. For test code, you can set a custom time that will be used by the struct. 882 883 <table> 884 <tbody> 885 <tr><th>Avoid 🔥</th></tr> 886 <tr><td> 887 888 ```go 889 func (s *SomeType) IsExpired(created time.Time) bool { 890 // Code is hardly testable. 891 return time.Since(created) >= s.expiryDuration 892 } 893 ``` 894 895 </td></tr> 896 <tr><th>Better 🤓</th></tr> 897 <tr><td> 898 899 ```go 900 func (s *SomeType) IsExpired(created time.Time) bool { 901 // s.timeNow is time.Now on production, mocked in tests. 902 return created.Add(s.expiryDuration).Before(s.timeNow()) 903 } 904 ``` 905 906 </td></tr> 907 </tbody></table> 908 909 ## Enforced by Linters 910 911 This is the list of rules we ensure automatically. This section is for those who are curious why such linting rules were added or want similar ones in their Go project. 🤗 912 913 #### Avoid Prints 914 915 Never use `print`. Always use a passed `go-kit/log.Logger`. 916 917 Ensured [here](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/Makefile#L311). 918 919 #### Ensure Prometheus Metric Registration 920 921 It's very easy to forget to add Prometheus metrics (e.g a `prometheus.Counter`) into a `registry.MustRegister` function. To avoid this, we ensure all metrics are created via `promtest.With(r).New*` and we disallow the old type of registration. Read more about the problem [here](https://github.com/thanos-io/thanos/issues/2102). 922 923 Ensured [here](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/Makefile#L308). 924 925 #### go vet 926 927 Standard Go vet is quite strict, but for a good reason. Always vet your Go code! 928 929 Ensured [here](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/Makefile#L313). 930 931 #### golangci-lint 932 933 [golangci-lint](https://github.com/golangci/golangci-lint) is an amazing tool that allows running a set of different custom linters from the Go community against your code. Give it a star and use it. (: 934 935 Ensured [here](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/Makefile#L315) with [those linters](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/.golangci.yml#L31) enabled. 936 937 #### misspell 938 939 Misspell is amazing, it catches typos in comments and docs. 940 941 No Grammarly plugin for this yet ): (We wish). 942 943 Ensured [here](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/Makefile#L#300), using [golangci-lint](https://github.com/golangci/golangci-lint) / [misspell](https://github.com/client9/misspell). 944 945 #### Comments Should be Full Sentences 946 947 All comments should be full sentences. They should start with an uppercase letter and end with a period. 948 949 Ensured [here](https://github.com/thanos-io/thanos/blob/40526f52f54d4501737e5246c0e71e56dd7e0b2d/Makefile#L300) using [golangci-lint](https://github.com/golangci/golangci-lint) / [godot](https://github.com/tetafro/godot). 950 951 # Bash 952 953 Overall try to NOT use bash. For scripts longer than 30 lines, consider writing it in Go as we did [here](https://github.com/thanos-io/thanos/blob/55cb8ca38b3539381dc6a781e637df15c694e50a/scripts/copyright/copyright.go). 954 955 If you have to, we follow the Google Shell style guide: https://google.github.io/styleguide/shellguide.html Ensured [here](https://github.com/thanos-io/thanos/blob/040b69b0b7c8e1be3890054bcb16389fa975eb45/Makefile#L165) using [shfmt](https://github.com/mvdan/sh). We also use [shellcheck](https://github.com/koalaman/shellcheck) to check any script errors.