github.com/ipld/go-ipld-prime@v0.21.0/schema/gen/go/HACKME_testing.md (about)

     1  Testing Generated Code
     2  ======================
     3  
     4  Getting nice testing scenarios for generated code is tricky.
     5  
     6  There are three major phases of testing we can do:
     7  
     8  1. unit tests on the codegen tooling itself
     9  2. tests that the emitted generated code can compile
    10  3. behavioral tests on the emitted generated code
    11  
    12  Groups 1 and 2 are pretty easy (and we don't have a lot of group 1,
    13  because group 2 covers it pretty nicely).
    14  
    15  Group 3, however, is tricky.
    16  The rest of the document will talk more about this kind of testing.
    17  
    18  
    19  Behavioral Tests
    20  ----------------
    21  
    22  ### Behavioral tests are run via plugins
    23  
    24  This package does some fancy footwork with the golang `plugin` system
    25  and `go build -buildmode=plugin` in order to compile and load the
    26  generated code into the same memory as the test process,
    27  thereby letting us do behavioral tests on the gen'd code quite seamlessly.
    28  
    29  This does have some downsides -- namely, the `plugin` system requires
    30  the use of `cgo`.  This means you'll need more things installed on your
    31  host system than just the go toolchain -- you might need `gcc` and friends too.
    32  The `plugin` system also (at time of writing) doesn't support windows.
    33  You can skip the behavioral tests if this is a problem: see the next section.
    34  
    35  ### Skipping the behavioral tests
    36  
    37  You can skip behavioral tests by adding a build tag of `"skipgenbehavtests"`.
    38  They'll also be automatically skipped if you're running in `"!cgo"` mode --
    39  however, the `go` tools don't automatically set `"!cgo"` just because it
    40  doesn't have enough tools, so you'll still need to be explicit about this.
    41  
    42  The ability of the generated code to be compiled will still be checked,
    43  even if the behavioral tests are skipped.
    44  
    45  You can grep through your test output for "bhvtest=skip" to see at-a-glance
    46  if the behavioral tests are being skipped.
    47  
    48  ### Plugins don't jive with race detection
    49  
    50  Long story short: if running tests with the race detector, skip the gen tests.
    51  Any `go test -race` is going to need `go test -race -tags 'skipgenbehavtests'`.
    52  
    53  The go plugin system requires the plugin and host process have the same "runtime".
    54  The way '-race' works makes for an effectively distinct runtime.
    55  
    56  ### Alternatives to plugins
    57  
    58  Are there other ways this could be done?  Well, surely.  There always are.
    59  
    60  Invoking 'go test' as a subprocess is usually central to alternative ideas,
    61  but this has several downsides I haven't yet figured out how to counter:
    62  
    63  - it tends to result in very difficult-to-wrangle output;
    64  - it ends up imposing a lot of constraints on file organization,
    65    which in turn makes writing tests into a very high-friction endeavour;
    66  - passing down flags to the test process (e.g., '-v' and friends)
    67    begins to require additional infrastructure;
    68  - some flags such as '-run' are even yet more difficult to pass down usefully;
    69  - every single behavioral test has to have an exported top-level function,
    70    making some things that are trivial with closures now difficult...
    71  - You get the idea.
    72  
    73  You can see some attempts around
    74  commit 79de0e26469f0d2899c813a2c70d921fe5946f23 and its halfdozen or so
    75  parents; remarks can be found in the git commit messages.
    76  
    77  There are probably yet more variations in ways files and functions could
    78  be reorganized, particularly to minimize the downsides of the file and
    79  package splitting requirements, but if you're looking at this scenario and
    80  wanting to propose one... Do read those commits to avoid getting into a
    81  Chesterton's Fence situation, and kindly try it before proposing it.
    82  Not all of the hurdles are immediately obvious.
    83  
    84  ### Plugins are only used in testing
    85  
    86  This might not need a clarification statement, but just in case it does:
    87  **plugins** (and by extention, cgo) **are not necessary**
    88  for **doing** codegen nor for **using** the resulting generated code.
    89  They are _only_ used for our testing of the codegen tooling
    90  (and specifically, at that, for the behavioral tests).
    91  We would not foist a dependency like cgo on codegen users.