github.com/iqoqo/nomad@v0.11.3-0.20200911112621-d7021c74d101/e2e/framework/doc.go (about)

     1  /*
     2  Package framework implements a model for developing end-to-end test suites. The
     3  model includes a top level Framework which TestSuites can be added to. TestSuites
     4  include conditions under which the suite will run and a list of TestCase
     5  implementations to run. TestCases can be implemented with methods that run
     6  before/after each and all tests.
     7  
     8  Writing Tests
     9  
    10  Tests follow a similar patterns as go tests. They are functions that must start
    11  with 'Test' and instead of a *testing.T argument, a *framework.F is passed and
    12  they must have a receiver that implements the TestCase interface.
    13  A crude example as follows:
    14  
    15  	// foo_test.go
    16  	type MyTestCase struct {
    17  		framework.TC
    18  	}
    19  
    20  	func (tc *MyTestCase) TestMyFoo(f *framework.F) {
    21  		f.T().Log("bar")
    22  	}
    23  
    24  	func TestCalledFromGoTest(t *testing.T){
    25  		framework.New().AddSuites(&framework.TestSuite{
    26  			Component:   "foo",
    27  			Cases: []framework.TestCase{
    28  				new(MyTestCase),
    29  			},
    30  		}).Run(t)
    31  	}
    32  
    33  Test cases should embed the TC struct which satisfies the TestCase interface.
    34  Optionally a TestCase can also implement the Name() function which returns
    35  a string to name the test case. By default the name is the name of the struct
    36  type, which in the above example would be "MyTestCase"
    37  
    38  Test cases may also optionally implement additional interfaces to define setup
    39  and teardown logic:
    40  
    41  	BeforeEachTest
    42  	AfterEachTest
    43  	BeforeAllTests
    44  	AfterAllTests
    45  
    46  The test case struct allows you to setup and teardown state in the struct that
    47  can be consumed by the tests. For example:
    48  
    49  	type ComplexNomadTC struct {
    50  		framework.TC
    51  		jobID string
    52  	}
    53  
    54  	func (tc *ComplexNomadTC) BeforeEach(f *framework.F){
    55  		// Do some complex job setup with a unique prefix string
    56  		jobID, err := doSomeComplexSetup(tc.Nomad(), f.ID())
    57  		f.NoError(err)
    58  		f.Set("jobID", jobID)
    59  	}
    60  
    61  	func (tc *ComplexNomadTC) TestSomeScenario(f *framework.F){
    62  		jobID := f.Value("jobID").(string)
    63  		doTestThingWithJob(f, tc.Nomad(), jobID)
    64  	}
    65  
    66  	func (tc *ComplexNomadTC) TestOtherScenario(f *framework.F){
    67  		jobID := f.Value("jobID").(string)
    68  		doOtherTestThingWithJob(f, tc.Nomad(), jobID)
    69  	}
    70  
    71  	func (tc *ComplexNomadTC) AfterEach(f *framework.F){
    72  		jobID := f.Value("jobID").(string)
    73  		_, _, err := tc.Nomad().Jobs().Deregister(jobID, true, nil)
    74  		f.NoError(err)
    75  	}
    76  
    77  As demonstrated in the previous example, TC also exposes functions that return
    78  configured api clients including Nomad, Consul and Vault. If Consul or Vault
    79  are not provisioned their respective getter functions will return nil.
    80  
    81  Testify Integration
    82  
    83  Test cases expose a T() function to fetch the current *testing.T context.
    84  While this means the author is able to most other testing libraries,
    85  github.com/stretch/testify is recommended and integrated into the framework.
    86  The TC struct also embeds testify assertions that are preconfigured with the
    87  current testing context. Additionally TC comes with a Require() method that
    88  yields a testify Require if that flavor is desired.
    89  
    90  	func (tc *MyTestCase) TestWithTestify() {
    91  		err := someErrFunc()
    92  		tc.NoError(err)
    93  		// Or tc.Require().NoError(err)
    94  	}
    95  
    96  Parallelism
    97  
    98  The test framework honors go test's parallel feature under certain conditions.
    99  A TestSuite can be created with the Parallel field set to true to enable
   100  parallel execution of the test cases of the suite. Tests within a test case
   101  will be executed sequentially unless f.T().Parallel() is called. Note that if
   102  multiple tests are to be executed in parallel, access to TC is not syncronized.
   103  The *framework.F offers a way to store state between before/after each method if
   104  desired.
   105  
   106  	func (tc *MyTestCase) BeforeEach(f *framework.F){
   107  		jobID, _ := doSomeComplexSetup(tc.Nomad(), f.ID())
   108  		f.Set("jobID", jobID)
   109  	}
   110  
   111  	func (tc *MyTestCase) TestParallel(f *framework.F){
   112  		f.T().Parallel()
   113  		jobID := f.Value("jobID").(string)
   114  	}
   115  
   116  Since test cases have the potential to work with a shared Nomad cluster in parallel
   117  any resources created or destroyed must be prefixed with a unique identifier for
   118  each test case. The framework.F struct exposes an ID() function that will return a
   119  string that is unique with in a test. Therefore, multiple tests with in the case
   120  can reliably create unique IDs between tests and setup/teardown. The string
   121  returned is 8 alpha numeric characters.
   122  
   123  */
   124  package framework