github.com/CycloneDX/sbom-utility@v0.16.0/sbom-validation-tests.md (about)

     1  [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
     2  
     3  # sbom-utility
     4  
     5  ## SBOM validation tests
     6  
     7  Validation tests are logically grouped by files as well as categories.
     8  
     9  ---
    10  
    11  ### validate_test.go
    12  
    13  The tests in this section are all found in [`cmd/validate_test.go`](cmd/validate_test.go).
    14  
    15  #### Input file tests
    16  
    17  Assures the utility properly handles invalid values on the `--input` (or `-i`) flag.
    18  
    19  | Test name | Description | Test input file | Expected results | Notes |
    20  | :-- | :-- | :-- | :-- | :-- |
    21  | TestValidateInvalidInputFileLoad | Test for invalid input file provided on the `-i` flag | TEST_INPUT_FILE_NON_EXISTENT | `fs.PathError`  |  |
    22  
    23  ---
    24  
    25  #### JSON syntax error tests
    26  
    27  Assures the utility properly handles different kinds of syntax errors in JSON documents (i.e., SBOMs) with the expected location (offset).
    28  
    29  **Note** Syntax error tests SHOULD return error type `encoding/json.SyntaxError`
    30  
    31  | Test name | Description | Syntax Error | Test input file |
    32  | :-- | :-- | :-- | :-- |
    33  |  TestValidateSyntaxErrorCdx13Test1 | Missing closing `}` bracket on `metadata` property  | "invalid character '{' after object key" | [test/cyclonedx/cdx-1-3-syntax-err-1.json](test/cyclonedx/cdx-1-3-syntax-err-1.json) |
    34  | TestValidateSyntaxErrorCdx13Test2 | Missing `:` separating `"properties"` key from array value `[` | "invalid character '[' after object key" | [test/cyclonedx/cdx-1-3-syntax-err-2.json](test/cyclonedx/cdx-1-3-syntax-err-2.json) |
    35  
    36  ---
    37  
    38  #### Custom schema tests
    39  
    40  Test custom schema validation (i.e., schemas provided using the `--force` flag).
    41  
    42  | Test name | Description | Schema file | Test input file | Expected results |
    43  | :-- | :-- | :-- | :-- | :-- |
    44  | TestValidateForceCustomSchemaCdx13 | Force validation against a "custom" schema with compatible format (CDX) and version (1.3) | TEST_SCHEMA_CDX_1_3_CUSTOM | TEST_CDX_1_3_MATURITY_BASE | *valid* |
    45  | TestValidateForceCustomSchemaCdx14 | Force validation against a "custom" schema with compatible format (CDX) and version (1.4) | TEST_SCHEMA_CDX_1_4_CUSTOM | TEST_CDX_1_4_MATURITY_BASE | *valid* |
    46  | TestValidateForceCustomSchemaCdxSchemaOlder | Force validation using schema with compatible format, but older version than the SBOM version | TEST_SCHEMA_CDX_1_3_CUSTOM | TEST_CDX_1_4_MATURITY_BASE | *valid* |
    47  
    48  ---
    49  
    50  ### validate_config_test.go
    51  
    52  The tests in this section are all found in [`cmd/validate_config_test.go`](cmd/validate_test.go).
    53  
    54  #### Configuration tests
    55  
    56  These tests verify that errors related to the `config.json` file entries (e.g., lookup of undefined formats, versions or variants) are returned properly.
    57  
    58  | Test name | Description | Expected results | Test input file | Notes |
    59  | :-- | :-- | :-- | :-- | :-- |
    60  | TestValidateConfigInvalidFormatKey | Error if SBOM "format" (key) undefined. |  `UnsupportedFormatError` | [test/config/test-base-invalid-format-key-foo.json](test/config/test-base-invalid-format-key-foo.json) | |
    61  | TestValidateConfigInvalidVersion         | Error if SBOM schema "version" not found (invalid) for a defined format. |  `UnsupportedSchemaError` |  [test/cyclonedx/cdx-1-x-test-invalid-spec-version.json](test/cyclonedx/cdx-1-x-test-invalid-spec-version.json) | |
    62  | TestValidateConfigInvalidVariant         | Error if SBOM schema "variant" not found (invalid) for a defined format and version. |  `UnsupportedSchemaError` | [test/cyclonedx/cdx-1-4-min-required.json](test/cyclonedx/cdx-1-4-min-required.json) | Reuse existing test file with valid format and version as `variant` value will not be found |
    63  | TestValidateConfigCDXBomFormatInvalid    | CDX `bomFormat` key value is invalid. | `UnsupportedFormatError` | [test/config/test-cdx-bom-format-invalid.json](test/config/test-cdx-bom-format-invalid.json) | |
    64  | TestValidateConfigCDXBomFormatMissing    | CDX `bomFormat` key is missing. | `UnsupportedFormatError` | [test/config/test-cdx-bom-format-missing.json](test/config/test-cdx-bom-format-missing.json) | |
    65  | TestValidateConfigCDXSpecVersionMissing  | CDX `specVersion` key is missing. | `UnsupportedSchemaError` | [test/config/test-cdx-spec-version-missing.json](test/config/test-cdx-spec-version-missing.json) | |
    66  | TestValidateConfigSPDXSpdxIdInvalid      | SPDX `SPDXID` key is invalid. | `UnsupportedFormatError` | [test/config/test-spdx-spdx-id-invalid.json](test/config/test-spdx-spdx-id-invalid.json) | |
    67  | TestValidateConfigSPDXSpdxVersionInvalid | SPDX `spdxVersion` key is invalid. | `UnsupportedSchemaError` | [test/config/test-spdx-spdx-version-missing.json](test/config/test-spdx-spdx-version-missing.json) | |
    68  
    69  ---
    70  
    71  #### CycloneDX Minimum Requirements tests
    72  
    73  | Test name | Description | Expected results | Test input file | Notes |
    74  | :-- | :-- | :-- | :-- | :-- |
    75  | TestValidateCdx13MinRequiredBasic |  |  |  |  |
    76  | TestValidateCdx14MinRequiredBasic |  |  |  |  |
    77  
    78  ---
    79  
    80  #### CycloneDX Example tests
    81  
    82  | Test name | Description | Expected results | Test input file | Notes |
    83  | :-- | :-- | :-- | :-- | :-- |
    84  | TestValidateExampleCdx14UseCaseAssembly |  |  |  |  |
    85  | TestValidateExampleCdx14UseCaseAuthenticityJsf |  |  |  |  |
    86  | TestValidateExampleCdx14UseCaseComponentKnownVulnerabilities |  |  |  |  |
    87  | TestValidateExampleCdx14UseCaseCompositionAndCompleteness |  |  |  |  |
    88  | TestValidateExampleCdx14UseCaseDependencyGraph |  |  |  |  |
    89  | TestValidateExampleCdx14UseCaseExternalReferences |  |  |  |  |
    90  | TestValidateExampleCdx14UseCaseIntegrityVerification |  |  |  |  |
    91  | TestValidateExampleCdx14UseCaseInventory |  |  |  |  |
    92  | TestValidateExampleCdx14UseCaseLicenseCompliance |  |  |  |  |
    93  | TestValidateExampleCdx14UseCaseOpenChainConformance |  |  |  |  |
    94  | TestValidateExampleCdx14UseCasePackageEvaluation |  |  |  |  |
    95  | TestValidateExampleCdx14UseCasePackagingDistribution |  |  |  |  |
    96  | TestValidateExampleCdx14UseCasePedigree |  |  |  |  |
    97  | TestValidateExampleCdx14UseCaseProvenance |  |  |  |  |
    98  | TestValidateExampleCdx14UseCaseSecurityAdvisories |  |  |  |  |
    99  | TestValidateExampleCdx14UseCaseServiceDefinition |  |  |  |  |
   100  | TestValidateExampleCdx14UseCaseVulnerabilityExploitation |  |  |  |  |
   101  | TestValidateExampleCdx14UseCaseVulnerabilityRemediation |  |  |  |  |
   102  | TestValidateExampleBomCdx12NpmJuiceShop |  |  |  |  |
   103  | TestValidateExampleBomCdx13Laravel |  |  |  |  |
   104  | TestValidateExampleSaaSBomCdx14ApiGatewayDatastores |  |  |  |  |