k8s.io/test-infra@v0.0.0-20240520184403-27c6b4c223d8/config/jobs/kubernetes/sig-node/README.md (about)

     1  ## Signode CI Test Coverage Guidance
     2  
     3  
     4  ## Variables
     5  
     6  
     7  <table>
     8    <tr>
     9     <td><strong>Variables</strong>
    10     </td>
    11     <td><strong>Values</strong>
    12     </td>
    13    </tr>
    14    <tr>
    15     <td>cri
    16     </td>
    17     <td>Containerd, CRI-O
    18     </td>
    19    </tr>
    20    <tr>
    21     <td>cgroup
    22     </td>
    23     <td>cgroupv1, cgroupv2
    24     </td>
    25    </tr>
    26    <tr>
    27     <td>os
    28     </td>
    29     <td>cos, ubuntu, fedora
    30     </td>
    31    </tr>
    32    <tr>
    33     <td>release branch
    34     </td>
    35     <td>N-3, N-2, N-1, N, main
    36     </td>
    37    </tr>
    38  </table>
    39  
    40  
    41  
    42  ## Naming Convention
    43  
    44  (pull|ci)-{$cri}-{$cgroup}-{$os}-{$test-type}-release-{$release-branch}
    45  
    46  **Note:** Specify variable value in the name only if it isn’t set to default in a test tab.
    47  
    48  
    49  ## Test Tabs
    50  
    51  #### 1. **sig-node-default (i.e. rename sig-node-release-blocking)**
    52  
    53  <table>
    54    <tr>
    55     <td>
    56  <strong>Variables</strong>
    57     </td>
    58     <td><strong>Default Values</strong>
    59     </td>
    60    </tr>
    61    <tr>
    62     <td>cri
    63     </td>
    64     <td>containerd,  CRI-O
    65     </td>
    66    </tr>
    67    <tr>
    68     <td>cgroup
    69     </td>
    70     <td>Cgroupv2
    71     </td>
    72    </tr>
    73    <tr>
    74     <td>os
    75     </td>
    76     <td>COS, ubuntu, fedora (create one instance for each OS in a single test job)
    77     </td>
    78    </tr>
    79    <tr>
    80     <td>release branch
    81     </td>
    82     <td>N-3, N-2, N-1, N, main
    83     </td>
    84    </tr>
    85  </table>
    86  
    87  
    88  **Test Types**
    89  * Node Conformance
    90  * Serial (Skip flaky, alpha and beta)
    91  * NodeFeature (Skip flaky, alpha and beta)
    92  
    93  **Additional tests**
    94  * Node Conformance test for CRI-O & cgroupv1
    95  
    96  **Test Names as per naming convention**
    97  
    98  {cri}-{cgroup}-{os}-{test-type}-release-{release-branch}
    99  
   100  * Node Conformance
   101      * node-conformance-release-(N-3)
   102      * node-conformance-release-(N-2)
   103      * node-conformance-release-(N-1)
   104      * node-conformance-release-(N)
   105      * node-conformance-release-main
   106  * Node Feature
   107      * node-feature-release-(N-3)
   108      * node-feature-release-(N-2)
   109      * node-feature-release-(N-1)
   110      * node-feature-release-(N)
   111      * node-feature-release-main
   112  * Serial
   113      * serial-release-(N-3)
   114      * serial-release-(N-2)
   115      * serial-release-(N-1)
   116      * serial-release-(N)
   117      * serial-release-main
   118  * Additional Node Conformance CRI-O cgroupv1 test
   119      * crio-cgroupv1-node-conformance
   120  
   121  **Release responsibility for new version (with k8s release cycle)**
   122  * Add a new image for a new K8s version.
   123      * Ensure COS and Ubuntu versions are set to K8s X default OS combination same as supported by managed kubernetes distributions AKS, EKS, GKE, etc. We can start with GKE supported combinations for some coverage, and then add for other distros when needed.
   124  * Update the release branches for the tests. Add tests against the new release branch, and remove tests against N-3 branch.
   125  
   126  #### 2. **sig-node-kubelet**
   127  
   128  <table>
   129    <tr>
   130     <td>
   131  <strong>Variables</strong>
   132     </td>
   133     <td><strong>Default Values</strong>
   134     </td>
   135    </tr>
   136    <tr>
   137     <td>cri
   138     </td>
   139     <td>containerd
   140     </td>
   141    </tr>
   142    <tr>
   143     <td>cgroup
   144     </td>
   145     <td>cgroupv2
   146     </td>
   147    </tr>
   148    <tr>
   149     <td>os
   150     </td>
   151     <td>COS
   152     </td>
   153    </tr>
   154    <tr>
   155     <td>release branch
   156     </td>
   157     <td>main
   158     </td>
   159    </tr>
   160  </table>
   161  
   162  
   163  **Test Types**
   164  * Node Conformance with all alpha features enabled.
   165  * Node Conformance with all beta features enabled.
   166  * Node Conformance with all alpha disabled.
   167  * Node Conformance with all beta disabled
   168  * Tests covering special features like in place pod vertical scaling. 
   169      * Features that require special node setup like swap, eviction, device plugin, huge pages, etc.
   170      * Features that require special K8s configuration e.g. Kubelet Credential Provider, lock contention, etc.
   171      * Feature tests specific to a cloud provider eg. GCP Credential Provider.
   172      * Version skew tests.
   173  
   174  **Test Names as per naming convention**
   175  * Unique names to identify the feature.
   176  * Follow {cri}-{cgroup}-{os}-{test-type|feature-name}-release-{release-branch} for ordering the variables for non-default values in test names i.e. If you’d like to add a test specific to a CRI, say CRIO and cgroupv2 on Fedora OS for main branch (which is default branch for this tab, the test name would be `crio-cgroup2-fedora-feature-name`.
   177  
   178  Examples: node-conformance-alpha-features-enabled, node-conformance-alpha-features-disabled, kubelet-credential-provider, memory-swap, version-skew-1-25-on-1-27, and so on.
   179  
   180  **Release responsibility for new version**
   181  * Add Version Skew tests for a new release, and remove tests for N-3 version.
   182  
   183  #### 3. **sig-node-containerd**
   184  
   185  **Test Types**
   186  * Node Features against containerd main, and for combinations of K8s X containerd versions supported by managed kubernetes distributions AKS, EKS, GKE, etc. We can start with GKE supported combinations for some coverage, and then add for other distros when needed.
   187  * Node Conformance  against containerd main, and for combinations of K8s X containerd versions supported by managed kubernetes distributions.
   188  * Serial  against containerd main, and for combinations of K8s X containerd versions supported by managed kubernetes distributions.
   189  * Containerd e2e tests for all containerd versions.
   190  * Containerd Build tests for all containerd versions.