github.com/snyk/vervet/v3@v3.7.0/README.md (about) 1 # vervet 2 3 Vervet is an HTTP API version lifecycle management tool, allowing APIs to be designed, developed, versioned and released from [resources](https://github.com/snyk/sweater-comb/blob/main/docs/intro.md#resources) independently and concurrently. 4 5 In a large organization, there might be many teams involved in delivering a large API -- such as at [Snyk](https://snyk.io) where Vervet was developed. 6 7 Within a single small team, there is still often a need to simultaneously try new things in parts of an API while maintaining stability. 8 9 While Vervet was developed in the context of a RESTful API, Vervet can be used with any HTTP API expressed in OpenAPI 3 -- even if it does not adhere to strict REST principles. 10 11 ### [API Versioning](https://github.com/snyk/sweater-comb/blob/main/docs/version.md) 12 13 To summarize the API versioning supported by Vervet: 14 15 #### What is versioned? 16 Resource versions are defined in OpenAPI 3, as if the resource were a standalone service. 17 18 #### How are resource version specs organized? 19 Resources are organized in a standard directory structure by release date, using OpenAPI extensions to define lifecycle concepts like stability. 20 21 #### How does versioning work? 22 * Resources are versioned independently by date and stability, with a well-defined deprecation and sunsetting policy. 23 * Additive, non-breaking changes can be made to released versions. Breaking changes trigger a new version. 24 * New versions deprecate and sunset prior versions, on a timeline determined by the stability level. 25 26 [Read more about API versioning](https://github.com/snyk/sweater-comb/blob/main/docs/version.md). 27 28 ## Features 29 30 A brief tour of Vervet's features. 31 32 ### Compilation 33 34 Vervet compiles the OpenAPI spec of each resource version into a series of OpenAPI specifications that describe the entire application, at each distinct release in its underlying parts. 35 36 Given a directory structure of resource versions, each defined by an OpenAPI spec as if it were an independent service: 37 38 ``` 39 $ tree resources 40 resources 41 ├── _examples 42 │ └── hello-world 43 │ ├── 2021-06-01 44 │ │ └── spec.yaml 45 │ ├── 2021-06-07 46 │ │ └── spec.yaml 47 │ └── 2021-06-13 48 │ └── spec.yaml 49 └── projects 50 └── 2021-06-04 51 └── spec.yaml 52 ``` 53 54 and a Vervet project configuration that instructs how to put them together: 55 56 ```yml 57 $ cat .vervet.yaml 58 apis: 59 my-api: 60 resources: 61 - path: 'resources' 62 output: 63 path: 'versions' 64 ``` 65 66 `vervet compile` aggregates these resources' individual OpenAPI specifications to describe the entire service API _at each distinct version date and stability level_ from its component parts. 67 68 ``` 69 $ tree versions 70 versions/ 71 ├── 2021-06-01 72 │ ├── spec.json 73 │ └── spec.yaml 74 ├── 2021-06-01~beta 75 │ ├── spec.json 76 │ └── spec.yaml 77 ├── 2021-06-01~experimental 78 │ ├── spec.json 79 │ └── spec.yaml 80 ├── 2021-06-04 81 │ ├── spec.json 82 │ └── spec.yaml 83 ├── 2021-06-04~beta 84 │ ├── spec.json 85 │ └── spec.yaml 86 ├── 2021-06-04~experimental 87 │ ├── spec.json 88 │ └── spec.yaml 89 ├── 2021-06-07 90 │ ├── spec.json 91 │ └── spec.yaml 92 ├── 2021-06-07~beta 93 │ ├── spec.json 94 │ └── spec.yaml 95 ├── 2021-06-07~experimental 96 │ ├── spec.json 97 │ └── spec.yaml 98 ├── 2021-06-13 99 │ ├── spec.json 100 │ └── spec.yaml 101 ├── 2021-06-13~beta 102 │ ├── spec.json 103 │ └── spec.yaml 104 └── 2021-06-13~experimental 105 ├── spec.json 106 └── spec.yaml 107 ``` 108 109 ### Linting 110 111 Vervet is not an OpenAPI linter. It coordinates and frontends OpenAPI linting, allowing different rules to be applied to different parts of an API, or different stages of the compilation process (source component specs, output compiled specs). It also allows exceptions to be made to certain resource versions, so that new rules do not break already-released parts of the API. 112 113 Vervet currently supports linting OpenAPI specifications with: 114 * [Spectral](https://stoplight.io/open-source/spectral/) 115 * [Sweater Comb](https://github.com/snyk/sweater-comb), as a self-contained Docker image which combines a linter and custom opinionated rulesets. 116 117 Direct Spectral linting may be soon deprecated in favor of container-based linting. 118 119 ### Generation 120 121 Since Vervet models the composition and construction of an API, it is well positioned to coordinate code and artifact generation through templates. 122 123 Generators are defined in `.vervet.yaml`: 124 125 ```yml 126 generators: 127 version-readme: 128 scope: version 129 filename: "resources/{{ .Resource }}/{{ .Version }}/README" 130 template: ".vervet/templates/README.tmpl" 131 version-spec: 132 scope: version 133 filename: "resources/{{ .Resource }}/{{ .Version }}/spec.yaml" 134 template: ".vervet/templates/spec.yaml.tmpl" 135 ``` 136 137 In this case, generators produce a boilerplate OpenAPI specification containing HTTP methods to create, list, get, update, and delete a resource, and a nice README when a new resource version is created. OpenAPI specifications can be tedious to write from scratch; generators help developers focus on adding the content that matters most. 138 139 Generators are defined using [Go templates](https://pkg.go.dev/text/template). Template syntax is also used to express filename interpolation per resource, per version. 140 141 ```yml 142 apis: 143 my-api: 144 resources: 145 - path: 'resources' 146 generators: 147 - version-readme 148 - version-spec 149 output: 150 path: 'versions' 151 ``` 152 153 Generators are applied during lifecycle commands, such as creating a new resource version: 154 155 ``` 156 $ vervet version new my-api thing 157 $ tree resources 158 resources 159 └── thing 160 └── 2021-10-21 161 ├── README 162 └── spec.yaml 163 ``` 164 165 Generators support multiple stages. For example, once a boilerplate spec.yaml is generated, it can be fed into subsequent generators that produce code, API gateway configuration, Grafana dashboards, and HTTP load tests. 166 167 A more advanced example, ExpressJS controllers generated from each operation in a resource version OpenAPI spec: 168 169 ```yml 170 generators: 171 version-spec: 172 scope: version 173 filename: "resources/{{ .Resource }}/{{ .Version }}/spec.yaml" 174 template: ".vervet/templates/spec.yaml.tmpl" 175 version-controller: 176 scope: version 177 # `files:` generates a collection of files -- which itself is expressed as a 178 # YAML template. Keys in this YAML are the paths of the files to generate, 179 # whose values are the file contents. 180 files: |- 181 {{- $resource := .Resource -}} 182 {{- $version := .Version -}} 183 {{- range $path, $pathItem := .Data.Spec.paths -}} 184 {{- range $method, $operation := $pathItem -}} 185 {{- $operationId := $operation.operationId -}} 186 {{/* Construct a context object using the 'map' function */}} 187 {{- $ctx := map "Context" . "OperationId" $operationId }} 188 resources/{{ $resource }}/{{ $version }}/{{ $operationId }}.ts: |- 189 {{/* 190 Evaluate the template by including it with the necessary context. 191 The generator's template is included as "contents" from within the 192 `files:` template. 193 */}} 194 {{ include "contents" $ctx | indent 2 }} 195 {{ end }} 196 {{- end -}} 197 template: ".vervet/resource/version/controller.ts.tmpl" 198 data: 199 Spec: 200 # generated above in version-spec, accessible from within the `files:` 201 # template as `.Data.Spec`. 202 include: "resources/{{ .Resource }}/{{ .Version }}/spec.yaml" 203 apis: 204 my-api: 205 resources: 206 - path: 'resources' 207 generators: 208 # order is important 209 - version-spec 210 - version-controller 211 output: 212 path: 'versions' 213 ``` 214 215 In this case, a template is being applied per `operationId` in the `spec.yaml` generated in the prior step. `version-controller` produces a collection of files, a controller module per resource, per version, per operation. This is possible because generators are applied in the order they are declared on each set of resources. 216 217 ### Scaffolding 218 219 Just as generators automate the generation of artifacts as part of the versioning lifecycle, scaffolds are used to bootstrap a new greenfield Vervet API project with useful defaults: 220 221 * Vervet project configuration (`.vervet.yaml`) 222 * Directory structure and layout for API specifications 223 * Generator templates 224 * Linter rulesets 225 226 Scaffolds are great in a microservice/SOA self-service ecosystem, where new services may be created often, and need a set of sensible defaults to quickly get started. 227 228 ``` 229 $ mkdir my-new-service 230 $ cd my-new-service 231 $ vervet scaffold init ../vervet-api-scaffold/ 232 $ tree -a 233 . 234 ├── .vervet 235 │ ├── components 236 │ │ ├── common.yaml 237 │ │ ├── errors.yaml 238 │ │ ├── headers 239 │ │ │ └── headers.yaml 240 │ │ ├── parameters 241 │ │ │ ├── pagination.yaml 242 │ │ │ └── version.yaml 243 │ │ ├── responses 244 │ │ │ ├── 204.yaml 245 │ │ │ ├── 400.yaml 246 │ │ │ ├── 401.yaml 247 │ │ │ ├── 403.yaml 248 │ │ │ ├── 404.yaml 249 │ │ │ ├── 409.yaml 250 │ │ │ ├── 429.yaml 251 │ │ │ └── 500.yaml 252 │ │ ├── tag.yaml 253 │ │ ├── types.yaml 254 │ │ └── version.yaml 255 │ ├── openapi 256 │ │ └── spec.yaml 257 │ └── templates 258 │ ├── README.tmpl 259 │ └── spec.yaml.tmpl 260 ├── .vervet.yaml 261 └── api 262 ├── resources 263 └── versions 264 ``` 265 266 This scaffold sets up a new project with standard OpenAPI components that are referenced by resource OpenAPI boilerplate templates. New resources are generated already conforming to our [JSON API](https://github.com/snyk/sweater-comb/blob/main/docs/jsonapi.md) standards and paginated list operations. 267 268 ## Installation 269 270 ### NPM 271 272 npm install -g @snyk/vervet 273 274 NPM packaging adapted from https://github.com/manifoldco/torus-cli. 275 276 ### Source 277 278 Go >= 1.16 required. 279 280 go build ./cmd/vervet 281 282 or 283 284 make build 285 286 ## Development 287 288 Vervet uses a reference set of OpenAPI documents in `testdata/resources` in 289 tests. CLI tests compare runtime compiled output with pre-compiled, expected 290 output in `testdata/output` to detect regressions. 291 292 When introducing changes that intentionally change the content of compiled 293 output: 294 295 * Run `go generate ./testdata` to update the contents of `testdata/output` 296 * Verify that the compiled output is correct 297 * Commit the changes to `testdata/output` in your proposed branch