github.com/CycloneDX/sbom-utility@v0.16.0/sbom-validation-tests.md (about) 1 [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) 2 3 # sbom-utility 4 5 ## SBOM validation tests 6 7 Validation tests are logically grouped by files as well as categories. 8 9 --- 10 11 ### validate_test.go 12 13 The tests in this section are all found in [`cmd/validate_test.go`](cmd/validate_test.go). 14 15 #### Input file tests 16 17 Assures the utility properly handles invalid values on the `--input` (or `-i`) flag. 18 19 | Test name | Description | Test input file | Expected results | Notes | 20 | :-- | :-- | :-- | :-- | :-- | 21 | TestValidateInvalidInputFileLoad | Test for invalid input file provided on the `-i` flag | TEST_INPUT_FILE_NON_EXISTENT | `fs.PathError` | | 22 23 --- 24 25 #### JSON syntax error tests 26 27 Assures the utility properly handles different kinds of syntax errors in JSON documents (i.e., SBOMs) with the expected location (offset). 28 29 **Note** Syntax error tests SHOULD return error type `encoding/json.SyntaxError` 30 31 | Test name | Description | Syntax Error | Test input file | 32 | :-- | :-- | :-- | :-- | 33 | TestValidateSyntaxErrorCdx13Test1 | Missing closing `}` bracket on `metadata` property | "invalid character '{' after object key" | [test/cyclonedx/cdx-1-3-syntax-err-1.json](test/cyclonedx/cdx-1-3-syntax-err-1.json) | 34 | TestValidateSyntaxErrorCdx13Test2 | Missing `:` separating `"properties"` key from array value `[` | "invalid character '[' after object key" | [test/cyclonedx/cdx-1-3-syntax-err-2.json](test/cyclonedx/cdx-1-3-syntax-err-2.json) | 35 36 --- 37 38 #### Custom schema tests 39 40 Test custom schema validation (i.e., schemas provided using the `--force` flag). 41 42 | Test name | Description | Schema file | Test input file | Expected results | 43 | :-- | :-- | :-- | :-- | :-- | 44 | TestValidateForceCustomSchemaCdx13 | Force validation against a "custom" schema with compatible format (CDX) and version (1.3) | TEST_SCHEMA_CDX_1_3_CUSTOM | TEST_CDX_1_3_MATURITY_BASE | *valid* | 45 | TestValidateForceCustomSchemaCdx14 | Force validation against a "custom" schema with compatible format (CDX) and version (1.4) | TEST_SCHEMA_CDX_1_4_CUSTOM | TEST_CDX_1_4_MATURITY_BASE | *valid* | 46 | TestValidateForceCustomSchemaCdxSchemaOlder | Force validation using schema with compatible format, but older version than the SBOM version | TEST_SCHEMA_CDX_1_3_CUSTOM | TEST_CDX_1_4_MATURITY_BASE | *valid* | 47 48 --- 49 50 ### validate_config_test.go 51 52 The tests in this section are all found in [`cmd/validate_config_test.go`](cmd/validate_test.go). 53 54 #### Configuration tests 55 56 These tests verify that errors related to the `config.json` file entries (e.g., lookup of undefined formats, versions or variants) are returned properly. 57 58 | Test name | Description | Expected results | Test input file | Notes | 59 | :-- | :-- | :-- | :-- | :-- | 60 | TestValidateConfigInvalidFormatKey | Error if SBOM "format" (key) undefined. | `UnsupportedFormatError` | [test/config/test-base-invalid-format-key-foo.json](test/config/test-base-invalid-format-key-foo.json) | | 61 | TestValidateConfigInvalidVersion | Error if SBOM schema "version" not found (invalid) for a defined format. | `UnsupportedSchemaError` | [test/cyclonedx/cdx-1-x-test-invalid-spec-version.json](test/cyclonedx/cdx-1-x-test-invalid-spec-version.json) | | 62 | TestValidateConfigInvalidVariant | Error if SBOM schema "variant" not found (invalid) for a defined format and version. | `UnsupportedSchemaError` | [test/cyclonedx/cdx-1-4-min-required.json](test/cyclonedx/cdx-1-4-min-required.json) | Reuse existing test file with valid format and version as `variant` value will not be found | 63 | TestValidateConfigCDXBomFormatInvalid | CDX `bomFormat` key value is invalid. | `UnsupportedFormatError` | [test/config/test-cdx-bom-format-invalid.json](test/config/test-cdx-bom-format-invalid.json) | | 64 | TestValidateConfigCDXBomFormatMissing | CDX `bomFormat` key is missing. | `UnsupportedFormatError` | [test/config/test-cdx-bom-format-missing.json](test/config/test-cdx-bom-format-missing.json) | | 65 | TestValidateConfigCDXSpecVersionMissing | CDX `specVersion` key is missing. | `UnsupportedSchemaError` | [test/config/test-cdx-spec-version-missing.json](test/config/test-cdx-spec-version-missing.json) | | 66 | TestValidateConfigSPDXSpdxIdInvalid | SPDX `SPDXID` key is invalid. | `UnsupportedFormatError` | [test/config/test-spdx-spdx-id-invalid.json](test/config/test-spdx-spdx-id-invalid.json) | | 67 | TestValidateConfigSPDXSpdxVersionInvalid | SPDX `spdxVersion` key is invalid. | `UnsupportedSchemaError` | [test/config/test-spdx-spdx-version-missing.json](test/config/test-spdx-spdx-version-missing.json) | | 68 69 --- 70 71 #### CycloneDX Minimum Requirements tests 72 73 | Test name | Description | Expected results | Test input file | Notes | 74 | :-- | :-- | :-- | :-- | :-- | 75 | TestValidateCdx13MinRequiredBasic | | | | | 76 | TestValidateCdx14MinRequiredBasic | | | | | 77 78 --- 79 80 #### CycloneDX Example tests 81 82 | Test name | Description | Expected results | Test input file | Notes | 83 | :-- | :-- | :-- | :-- | :-- | 84 | TestValidateExampleCdx14UseCaseAssembly | | | | | 85 | TestValidateExampleCdx14UseCaseAuthenticityJsf | | | | | 86 | TestValidateExampleCdx14UseCaseComponentKnownVulnerabilities | | | | | 87 | TestValidateExampleCdx14UseCaseCompositionAndCompleteness | | | | | 88 | TestValidateExampleCdx14UseCaseDependencyGraph | | | | | 89 | TestValidateExampleCdx14UseCaseExternalReferences | | | | | 90 | TestValidateExampleCdx14UseCaseIntegrityVerification | | | | | 91 | TestValidateExampleCdx14UseCaseInventory | | | | | 92 | TestValidateExampleCdx14UseCaseLicenseCompliance | | | | | 93 | TestValidateExampleCdx14UseCaseOpenChainConformance | | | | | 94 | TestValidateExampleCdx14UseCasePackageEvaluation | | | | | 95 | TestValidateExampleCdx14UseCasePackagingDistribution | | | | | 96 | TestValidateExampleCdx14UseCasePedigree | | | | | 97 | TestValidateExampleCdx14UseCaseProvenance | | | | | 98 | TestValidateExampleCdx14UseCaseSecurityAdvisories | | | | | 99 | TestValidateExampleCdx14UseCaseServiceDefinition | | | | | 100 | TestValidateExampleCdx14UseCaseVulnerabilityExploitation | | | | | 101 | TestValidateExampleCdx14UseCaseVulnerabilityRemediation | | | | | 102 | TestValidateExampleBomCdx12NpmJuiceShop | | | | | 103 | TestValidateExampleBomCdx13Laravel | | | | | 104 | TestValidateExampleSaaSBomCdx14ApiGatewayDatastores | | | | |