github.com/graybobo/golang.org-package-offline-cache@v0.0.0-20200626051047-6608995c132f/x/blog/content/generate.article (about) 1 Generating code 2 22 Dec 2014 3 Tags: programming, technical 4 5 Rob Pike 6 7 * Generating code 8 9 A property of universal computation—Turing completeness—is that a computer program can write a computer program. 10 This is a powerful idea that is not appreciated as often as it might be, even though it happens frequently. 11 It's a big part of the definition of a compiler, for instance. 12 It's also how the `go` `test` command works: it scans the packages to be tested, 13 writes out a Go program containing a test harness customized for the package, 14 and then compiles and runs it. 15 Modern computers are so fast this expensive-sounding sequence can complete in a fraction of a second. 16 17 There are lots of other examples of programs that write programs. 18 [[http://golang.org/cmd/yacc/][Yacc]], for instance, reads in a description of a grammar and writes out a program to parse that grammar. 19 The protocol buffer "compiler" reads an interface description and emits structure definitions, 20 methods, and other support code. 21 Configuration tools of all sorts work like this too, examining metadata or the environment 22 and emitting scaffolding customized to the local state. 23 24 Programs that write programs are therefore important elements in software engineering, 25 but programs like Yacc that produce source code need to be integrated into the build 26 process so their output can be compiled. 27 When an external build tool like Make is being used, this is usually easy to do. 28 But in Go, whose go tool gets all necessary build information from the Go source, there is a problem. 29 There is simply no mechanism to run Yacc from the go tool alone. 30 31 Until now, that is. 32 33 The [[http://blog.golang.org/go1.4][latest Go release]], 1.4, 34 includes a new command that makes it easier to run such tools. 35 It's called `go` `generate`, and it works by scanning for special comments in Go source code 36 that identify general commands to run. 37 It's important to understand that `go` `generate` is not part of `go` `build`. 38 It contains no dependency analysis and must be run explicitly before running `go` `build`. 39 It is intended to be used by the author of the Go package, not its clients. 40 41 The `go` `generate` command is easy to use. 42 As a warmup, here's how to use it to generate a Yacc grammar. 43 Say you have a Yacc input file called `gopher.y` that defines a grammar for your new language. 44 To produce the Go source file implementing the grammar, 45 you would normally invoke the standard Go version of Yacc like this: 46 47 go tool yacc -o gopher.go -p parser gopher.y 48 49 The `-o` option names the output file while `-p` specifies the package name. 50 51 To have `go` `generate` drive the process, in any one of the regular (non-generated) `.go` files 52 in the same directory, add this comment anywhere in the file: 53 54 //go:generate go tool yacc -o gopher.go -p parser gopher.y 55 56 This text is just the command above prefixed by a special comment recognized by `go` `generate`. 57 The comment must start at the beginning of the line and have no spaces between the `//` and the `go:generate`. 58 After that marker, the rest of the line specifies a command for `go` `generate` to run. 59 60 Now run it. Change to the source directory and run `go` `generate`, then `go` `build` and so on: 61 62 $ cd $GOPATH/myrepo/gopher 63 $ go generate 64 $ go build 65 $ go test 66 67 That's it. 68 Assuming there are no errors, the `go` `generate` command will invoke `yacc` to create `gopher.go`, 69 at which point the directory holds the full set of Go source files, so we can build, test, and work normally. 70 Every time `gopher.y` is modified, just rerun `go` `generate` to regenerate the parser. 71 72 For more details about how `go` `generate` works, including options, environment variables, 73 and so on, see the [[http://golang.org/s/go1.4-generate][design document]]. 74 75 Go generate does nothing that couldn't be done with Make or some other build mechanism, 76 but it comes with the `go` tool—no extra installation required—and fits nicely into the Go ecosystem. 77 Just keep in mind that it is for package authors, not clients, 78 if only for the reason that the program it invokes might not be available on the target machine. 79 Also, if the containing package is intended for import by `go` `get`, 80 once the file is generated (and tested!) it must be checked into the 81 source code repository to be available to clients. 82 83 Now that we have it, let's use it for something new. 84 As a very different example of how `go` `generate` can help, there is a new program available in the 85 `golang.org/x/tools` repository called `stringer`. 86 It automatically writes string methods for sets of integer constants. 87 It's not part of the released distribution, but it's easy to install: 88 89 $ go get golang.org/x/tools/cmd/stringer 90 91 Here's an example from the documentation for 92 [[http://godoc.org/golang.org/x/tools/cmd/stringer][`stringer`]]. 93 Imagine we have some code that contains a set of integer constants defining different types of pills: 94 95 package painkiller 96 97 type Pill int 98 99 const ( 100 Placebo Pill = iota 101 Aspirin 102 Ibuprofen 103 Paracetamol 104 Acetaminophen = Paracetamol 105 ) 106 107 For debugging, we'd like these constants to pretty-print themselves, which means we want a method with signature, 108 109 func (p Pill) String() string 110 111 It's easy to write one by hand, perhaps like this: 112 113 func (p Pill) String() string { 114 switch p { 115 case Placebo: 116 return "Placebo" 117 case Aspirin: 118 return "Aspirin" 119 case Ibuprofen: 120 return "Ibuprofen" 121 case Paracetamol: // == Acetaminophen 122 return "Paracetamol" 123 } 124 return fmt.Sprintf("Pill(%d)", p) 125 } 126 127 There are other ways to write this function, of course. 128 We could use a slice of strings indexed by Pill, or a map, or some other technique. 129 Whatever we do, we need to maintain it if we change the set of pills, and we need to make sure it's correct. 130 (The two names for paracetamol make this trickier than it might otherwise be.) 131 Plus the very question of which approach to take depends on the types and values: 132 signed or unsigned, dense or sparse, zero-based or not, and so on. 133 134 The `stringer` program takes care of all these details. 135 Although it can be run in isolation, it is intended to be driven by `go` `generate`. 136 To use it, add a generate comment to the source, perhaps near the type definition: 137 138 //go:generate stringer -type=Pill 139 140 This rule specifies that `go` `generate` should run the `stringer` tool to generate a `String` method for type `Pill`. 141 The output is automatically written to `pill_string.go` (a default we could override with the 142 `-output` flag). 143 144 Let's run it: 145 146 $ go generate 147 $ cat pill_string.go 148 // generated by stringer -type Pill pill.go; DO NOT EDIT 149 150 package pill 151 152 import "fmt" 153 154 const _Pill_name = "PlaceboAspirinIbuprofenParacetamol" 155 156 var _Pill_index = [...]uint8{0, 7, 14, 23, 34} 157 158 func (i Pill) String() string { 159 if i < 0 || i+1 >= Pill(len(_Pill_index)) { 160 return fmt.Sprintf("Pill(%d)", i) 161 } 162 return _Pill_name[_Pill_index[i]:_Pill_index[i+1]] 163 } 164 $ 165 166 Every time we change the definition of `Pill` or the constants, all we need to do is run 167 168 $ go generate 169 170 to update the `String` method. 171 And of course if we've got multiple types set up this way in the same package, 172 that single command will update all their `String` methods with a single command. 173 174 There's no question the generated method is ugly. 175 That's OK, though, because humans don't need to work on it; machine-generated code is often ugly. 176 It's working hard to be efficient. 177 All the names are smashed together into a single string, 178 which saves memory (only one string header for all the names, even if there are zillions of them). 179 Then an array, `_Pill_index`, maps from value to name by a simple, efficient technique. 180 Note too that `_Pill_index` is an array (not a slice; one more header eliminated) of `uint8`, 181 the smallest integer sufficient to span the space of values. 182 If there were more values, or there were negatives ones, 183 the generated type of `_Pill_index` might change to `uint16` or `int8`: whatever works best. 184 185 The approach used by the methods printed by `stringer` varies according to the properties of the constant set. 186 For instance, if the constants are sparse, it might use a map. 187 Here's a trivial example based on a constant set representing powers of two: 188 189 const _Power_name = "p0p1p2p3p4p5..." 190 191 var _Power_map = map[Power]string{ 192 1: _Power_name[0:2], 193 2: _Power_name[2:4], 194 4: _Power_name[4:6], 195 8: _Power_name[6:8], 196 16: _Power_name[8:10], 197 32: _Power_name[10:12], 198 ..., 199 } 200 201 func (i Power) String() string { 202 if str, ok := _Power_map[i]; ok { 203 return str 204 } 205 return fmt.Sprintf("Power(%d)", i) 206 } 207 208 209 In short, generating the method automatically allows us to do a better job than we would expect a human to do. 210 211 There are lots of other uses of `go` `generate` already installed in the Go tree. 212 Examples include generating Unicode tables in the `unicode` package, 213 creating efficient methods for encoding and decoding arrays in `encoding/gob`, 214 producing time zone data in the `time` package, and so on. 215 216 Please use `go` `generate` creatively. 217 It's there to encourage experimentation. 218 219 And even if you don't, use the new `stringer` tool to write your `String` methods for your integer constants. 220 Let the machine do the work.