github.com/graybobo/golang.org-package-offline-cache@v0.0.0-20200626051047-6608995c132f/x/blog/content/generate.article (about)

     1  Generating code
     2  22 Dec 2014
     3  Tags: programming, technical
     4  
     5  Rob Pike
     6  
     7  * Generating code
     8  
     9  A property of universal computation—Turing completeness—is that a computer program can write a computer program.
    10  This is a powerful idea that is not appreciated as often as it might be, even though it happens frequently.
    11  It's a big part of the definition of a compiler, for instance.
    12  It's also how the `go` `test` command works: it scans the packages to be tested,
    13  writes out a Go program containing a test harness customized for the package,
    14  and then compiles and runs it.
    15  Modern computers are so fast this expensive-sounding sequence can complete in a fraction of a second.
    16  
    17  There are lots of other examples of programs that write programs.
    18  [[http://golang.org/cmd/yacc/][Yacc]], for instance, reads in a description of a grammar and writes out a program to parse that grammar.
    19  The protocol buffer "compiler" reads an interface description and emits structure definitions,
    20  methods, and other support code.
    21  Configuration tools of all sorts work like this too, examining metadata or the environment
    22  and emitting scaffolding customized to the local state.
    23  
    24  Programs that write programs are therefore important elements in software engineering,
    25  but programs like Yacc that produce source code need to be integrated into the build
    26  process so their output can be compiled.
    27  When an external build tool like Make is being used, this is usually easy to do.
    28  But in Go, whose go tool gets all necessary build information from the Go source, there is a problem.
    29  There is simply no mechanism to run Yacc from the go tool alone.
    30  
    31  Until now, that is.
    32  
    33  The [[http://blog.golang.org/go1.4][latest Go release]], 1.4,
    34  includes a new command that makes it easier to run such tools.
    35  It's called `go` `generate`, and it works by scanning for special comments in Go source code
    36  that identify general commands to run.
    37  It's important to understand that `go` `generate` is not part of `go` `build`.
    38  It contains no dependency analysis and must be run explicitly before running `go` `build`.
    39  It is intended to be used by the author of the Go package, not its clients.
    40  
    41  The `go` `generate` command is easy to use.
    42  As a warmup, here's how to use it to generate a Yacc grammar.
    43  Say you have a Yacc input file called `gopher.y` that defines a grammar for your new language.
    44  To produce the Go source file implementing the grammar,
    45  you would normally invoke the standard Go version of Yacc like this:
    46  
    47  	go tool yacc -o gopher.go -p parser gopher.y
    48  
    49  The `-o` option names the output file while `-p` specifies the package name.
    50  
    51  To have `go` `generate` drive the process, in any one of the regular (non-generated) `.go` files
    52  in the same directory, add this comment anywhere in the file:
    53  
    54  	//go:generate go tool yacc -o gopher.go -p parser gopher.y
    55  
    56  This text is just the command above prefixed by a special comment recognized by `go` `generate`.
    57  The comment must start at the beginning of the line and have no spaces between the `//` and the `go:generate`.
    58  After that marker, the rest of the line specifies a command for `go` `generate` to run.
    59  
    60  Now run it. Change to the source directory and run `go` `generate`, then `go` `build` and so on:
    61  
    62  	$ cd $GOPATH/myrepo/gopher
    63  	$ go generate
    64  	$ go build
    65  	$ go test
    66  
    67  That's it.
    68  Assuming there are no errors, the `go` `generate` command will invoke `yacc` to create `gopher.go`,
    69  at which point the directory holds the full set of Go source files, so we can build, test, and work normally.
    70  Every time `gopher.y` is modified, just rerun `go` `generate` to regenerate the parser.
    71  
    72  For more details about how `go` `generate` works, including options, environment variables,
    73  and so on, see the [[http://golang.org/s/go1.4-generate][design document]].
    74  
    75  Go generate does nothing that couldn't be done with Make or some other build mechanism,
    76  but it comes with the `go` tool—no extra installation required—and fits nicely into the Go ecosystem.
    77  Just keep in mind that it is for package authors, not clients,
    78  if only for the reason that the program it invokes might not be available on the target machine.
    79  Also, if the containing package is intended for import by `go` `get`,
    80  once the file is generated (and tested!) it must be checked into the
    81  source code repository to be available to clients.
    82  
    83  Now that we have it, let's use it for something new.
    84  As a very different example of how `go` `generate` can help, there is a new program available in the
    85  `golang.org/x/tools` repository called `stringer`.
    86  It automatically writes string methods for sets of integer constants.
    87  It's not part of the released distribution, but it's easy to install:
    88  
    89  	$ go get golang.org/x/tools/cmd/stringer
    90  
    91  Here's an example from the documentation for
    92  [[http://godoc.org/golang.org/x/tools/cmd/stringer][`stringer`]].
    93  Imagine we have some code that contains a set of integer constants defining different types of pills:
    94  
    95  	package painkiller
    96  
    97  	type Pill int
    98  
    99  	const (
   100  		Placebo Pill = iota
   101  		Aspirin
   102  		Ibuprofen
   103  		Paracetamol
   104  		Acetaminophen = Paracetamol
   105  	)
   106  
   107  For debugging, we'd like these constants to pretty-print themselves, which means we want a method with signature,
   108  
   109  	func (p Pill) String() string
   110  
   111  It's easy to write one by hand, perhaps like this:
   112  
   113  	func (p Pill) String() string {
   114  		switch p {
   115  		case Placebo:
   116  			return "Placebo"
   117  		case Aspirin:
   118  			return "Aspirin"
   119  		case Ibuprofen:
   120  			return "Ibuprofen"
   121  		case Paracetamol: // == Acetaminophen
   122  			return "Paracetamol"
   123  		}
   124  		return fmt.Sprintf("Pill(%d)", p)
   125  	}
   126  
   127  There are other ways to write this function, of course.
   128  We could use a slice of strings indexed by Pill, or a map, or some other technique.
   129  Whatever we do, we need to maintain it if we change the set of pills, and we need to make sure it's correct.
   130  (The two names for paracetamol make this trickier than it might otherwise be.)
   131  Plus the very question of which approach to take depends on the types and values:
   132  signed or unsigned, dense or sparse, zero-based or not, and so on.
   133  
   134  The `stringer` program takes care of all these details.
   135  Although it can be run in isolation, it is intended to be driven by `go` `generate`.
   136  To use it, add a generate comment to the source, perhaps near the type definition:
   137  
   138  	//go:generate stringer -type=Pill
   139  
   140  This rule specifies that `go` `generate` should run the `stringer` tool to generate a `String` method for type `Pill`.
   141  The output is automatically written to `pill_string.go` (a default we could override with the
   142  `-output` flag).
   143  
   144  Let's run it:
   145  
   146  	$ go generate
   147  	$ cat pill_string.go
   148  	// generated by stringer -type Pill pill.go; DO NOT EDIT
   149  	
   150  	package pill
   151  	
   152  	import "fmt"
   153  	
   154  	const _Pill_name = "PlaceboAspirinIbuprofenParacetamol"
   155  	
   156  	var _Pill_index = [...]uint8{0, 7, 14, 23, 34}
   157  	
   158  	func (i Pill) String() string {
   159  		if i < 0 || i+1 >= Pill(len(_Pill_index)) {
   160  			return fmt.Sprintf("Pill(%d)", i)
   161  		}
   162  		return _Pill_name[_Pill_index[i]:_Pill_index[i+1]]
   163  	}
   164  	$
   165  
   166  Every time we change the definition of `Pill` or the constants, all we need to do is run
   167  
   168  	$ go generate
   169  
   170  to update the `String` method.
   171  And of course if we've got multiple types set up this way in the same package,
   172  that single command will update all their `String` methods with a single command.
   173  
   174  There's no question the generated method is ugly.
   175  That's OK, though, because humans don't need to work on it; machine-generated code is often ugly.
   176  It's working hard to be efficient.
   177  All the names are smashed together into a single string,
   178  which saves memory (only one string header for all the names, even if there are zillions of them).
   179  Then an array, `_Pill_index`, maps from value to name by a simple, efficient technique.
   180  Note too that `_Pill_index` is an array (not a slice; one more header eliminated) of `uint8`,
   181  the smallest integer sufficient to span the space of values.
   182  If there were more values, or there were negatives ones,
   183  the generated type of `_Pill_index` might change to `uint16` or `int8`: whatever works best.
   184  
   185  The approach used by the methods printed by `stringer` varies according to the properties of the constant set.
   186  For instance, if the constants are sparse, it might use a map.
   187  Here's a trivial example based on a constant set representing powers of two:
   188  
   189  	const _Power_name = "p0p1p2p3p4p5..."
   190  
   191  	var _Power_map = map[Power]string{
   192  		1:    _Power_name[0:2],
   193  		2:    _Power_name[2:4],
   194  		4:    _Power_name[4:6],
   195  		8:    _Power_name[6:8],
   196  		16:   _Power_name[8:10],
   197  		32:   _Power_name[10:12],
   198  		...,
   199  	}
   200  
   201  	func (i Power) String() string {
   202  		if str, ok := _Power_map[i]; ok {
   203  			return str
   204  		}
   205  		return fmt.Sprintf("Power(%d)", i)
   206  	}
   207  
   208  
   209  In short, generating the method automatically allows us to do a better job than we would expect a human to do.
   210  
   211  There are lots of other uses of `go` `generate` already installed in the Go tree.
   212  Examples include generating Unicode tables in the `unicode` package,
   213  creating efficient methods for encoding and decoding arrays in `encoding/gob`,
   214  producing time zone data in the `time` package, and so on.
   215  
   216  Please use `go` `generate` creatively.
   217  It's there to encourage experimentation.
   218  
   219  And even if you don't, use the new `stringer` tool to write your `String` methods for your integer constants.
   220  Let the machine do the work.