github.com/wmuizelaar/kpt@v0.0.0-20221018115725-bd564717b2ed/site/installation/migration.md (about) 1 ## Migrating from kpt `v0.39` to `v1.0` 2 3 - [Before you begin](#Before-you-begin) 4 - [What's new and changed?](#What's-new-and-changed?) 5 - [CLI changes](#CLI-changes) 6 - [Kptfile schema changes](#Kptfile-schema-changes) 7 - [`pkg`](#pkg) 8 - [`sync` merged with `update`](#sync-merged-with-update) 9 - [`cfg`](#cfg) 10 - [Changes to Setters](#Changes-to-Setters) 11 - [Setter validations deprecated](#Setter-validations-deprecated) 12 - [Auto setters deprecated](#Auto-setters-deprecated) 13 - [`fn`](#fn) 14 - [`run` is split into `eval` and `render`](#run-is-split-into-eval-and-render) 15 - [Function Config](#Function-Config) 16 - [Function Results](#Function-Results) 17 - [`live`](#live) 18 - [Migration steps](#Migration-steps) 19 - [For Package Publishers](#For-Package-Publishers) 20 - [Automated portion of migration](#Automated-portion-of-migration) 21 - [Changes made by the function](#Changes-made-by-the-function) 22 - [Manual portion of migration](#Manual-portion-of-migration) 23 - [For Package Consumers](#For-Package-Consumers) 24 - [Timeline](#Timeline) 25 26 kpt `v1.0` is going to be the latest major release of the kpt CLI. The 27 implementation of kpt `v1.0` has changed considerably from kpt `v0.39`. A rich 28 set of new features have been added as a result of the users’ input and 29 requirements. Some features have been deprecated or refactored in ways that make 30 them incompatible with kpt `v0.39`. Since these are backwards incompatible 31 changes, there should be a way for users to migrate/fix their existing kpt 32 packages which are compatible with `v0.39` version of kpt, to become compatible 33 with kpt `v1.0`. This document outlines the end to end migration journey of 34 users using a comprehensive kpt package example. 35 36 ## Before you begin 37 38 Please go through [installation instructions] for installing `v1.0` binary and 39 at least Chapter 1 and 2 of [The kpt Book] for understanding the basic model of 40 kpt `v1.0`. 41 42 ## What's new and changed? 43 44 ### CLI changes 45 46 To start with, all the commands in kpt `v1.0` will follow the consistent pattern 47 48 ``` 49 $ kpt <group> <command> <positional_args> [PKG_PATH | DIR | - STDIN] [flags] 50 ``` 51 52 Almost all the existing commands/features in kpt `v0.39` are also offered in kpt 53 `v1.0` but in a better and enhanced format. In kpt `v0.39`, `cfg` group had been 54 a dumping ground for many cli commands which don’t have a coherent story. `cfg` 55 is often confused with `pkg` and vice-versa. As a major step, we removed the 56 `cfg` subgroup and rearranged the functionality. Here are the one-liners for 57 each command group in kpt `v1.0`. 58 59 1. `pkg:` Commands for composing and describing packages. 60 2. `fn:` Commands for running functions for validation and customization. 61 3. `live:` Commands for interacting with a live cluster. 62 63 `PKG_PATH vs DIR:` Most of the commands accept only `PKG_PATH` as input which 64 means the input directory must have a valid Kptfile. Few commands can work on 65 just the simple `DIR` with resources. Commands operate on the current working 66 directory by default. 67 68 | [v0.39 Commands] | [v1.0 Commands] | 69 | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 70 | `kpt pkg get REPO_URI[.git]/PKG_PATH[@VERSION] LOCAL_DEST_DIRECTORY [flags]` | `kpt pkg get REPO_URI[.git]/PKG_PATH[@VERSION] [flags] [LOCAL_DEST_DIRECTORY]` <br> Fetch a remote package from a git subdirectory and writes it to a new local directory. | 71 | `kpt pkg init DIR [flags]` | `kpt pkg init [DIR] [flags]` <br> Initializes an existing empty directory as a kpt package by adding a Kptfile. | 72 | `kpt pkg update LOCAL_PKG_DIR[@VERSION] [flags]` | `kpt pkg update [PKG_PATH][@version] [flags]` <br> Pulls in upstream changes and merges them into a local package. | 73 | `kpt pkg fix DIR [flags]` | `kpt fn eval --image gcr.io/kpt-fn/fix:v0.2 --include-meta-resources` <br> Fix a local package which is using deprecated features. | 74 | `kpt pkg desc DIR [flags]` | Deprecated in favor of reading Kptfile directly | 75 | `kpt pkg diff DIR[@VERSION] [flags]` | `kpt pkg diff [PKG_PATH][@version] [flags]` <br> Display differences between upstream and local packages. | 76 | `kpt cfg fmt DIR/STDIN [flags]` | `kpt fn eval --image gcr.io/kpt-fn/format:v0.1` | 77 | `kpt cfg tree DIR/STDIN [flags]` | `kpt pkg tree [DIR] [flags]` <br> Displays resources, files and packages in a tree structure. | 78 | `kpt cfg cat DIR/STDIN [flags]` | `kpt fn source [DIR] -o unwrap` | 79 | `kpt fn run DIR/STDIN [flags]` | `kpt fn eval [DIR / -] [flags]` <br> Executes a single function on resources in a directory. <br> <br> `kpt fn render [PKG_PATH]` <br> Executes the pipeline of functions on resources in the package and writes the output to the local filesystem. | 80 | `kpt fn source DIR [flags]` | `kpt fn source [DIR] [flags]` <br> Reads resources from a local directory and writes them in Function Specification wire format to stdout. | 81 | `kpt fn sink DIR [flags]` | `kpt fn sink [DIR] [flags]` <br> Reads resources from stdin and writes them to a local directory. | 82 | `kpt fn export DIR [flags]` | Deprecated. | 83 | `kpt live init DIR [flags]` | `kpt live init [PKG_PATH] [flags]` <br> Initializes the package with the name, namespace and id of the resource that will keep track of the package inventory. | 84 | `kpt live apply DIR/STDIN [flags]` | `kpt live apply [PKG_PATH / -] [flags]` <br> Creates, updates and deletes resources in the cluster to make the remote cluster resources match the local package configuration. | 85 | `kpt live preview DIR/STDIN [flags]` | Deprecated. The functionality will be provided by `dry-run` flag in `apply` and `destroy` commands. | 86 | `kpt live destroy DIR/STDIN [flags]` | `kpt live destroy [PKG_PATH / -] [flags]` <br> Removes all files belonging to a package from the cluster. | 87 | `kpt live status DIR/STDIN [flags]` | `kpt live status [PKG_PATH / -] [flags]` <br> Shows the resource status for resources belonging to the package. | 88 | `kpt live diff DIR/STDIN [flags]` | Deprecated. The functionality will be provided by `dry-run` flag in `apply` and `destroy` commands. | 89 | `kpt cfg set DIR setter_name setter_value` | `kpt fn eval --image gcr.io/kpt-fn/apply-setters:v0.1 -- 'foo=bar' 'env=[dev, stage]'` | 90 | `kpt cfg create-setter DIR setter_name setter_value` | `kpt fn eval --image gcr.io/kpt-fn/search-replace:v0.1 -- 'by-value=nginx' 'put-comment=kpt-set: ${image}'` | 91 | `kpt cfg create-subst DIR subst_name --field-value nginx:1.7.1 --pattern ${image}:${tag} [flags]` | `kpt fn eval --image gcr.io/kpt-fn/search-replace:v0.1 -- 'by-value=nginx:1.7.1' 'put-comment=kpt-set: ${image}:${tag}'` | 92 | `kpt cfg delete-setter DIR setter_name` | `kpt fn eval --image gcr.io/kpt-fn/search-replace:v0.1 -- 'by-value=nginx' put-comment=''` | 93 | `kpt cfg delete-subst DIR subst_name [flags]` | `kpt fn eval --image gcr.io/kpt-fn/search-replace:0.1 -- 'by-value=nginx:1.7.1' put-comment=''` | 94 | `kpt cfg annotate DIR/STDIN [flags]` | `kpt fn eval --image gcr.io/kpt-fn/set-annotations:v0.1 -- 'name=foo' 'value=bar'` | 95 | `kpt cfg grep DIR/STDIN [flags]` | `kpt fn eval --image gcr.io/kpt-fn/search-replace:v0.1 -- 'by-value=foo' 'by-path=bar'` | 96 | `kpt cfg count DIR/STDIN [flags]` | Deprecated. | 97 98 ### Kptfile schema changes 99 100 The existing [v1alpha1 Kptfile] format/schema is not compatible with kpt `v1.0`. 101 Here is the schema for [v1 Kptfile] which is compatible with kpt `v1.0`. 102 103 1. The `packageMetaData` section in `v1alpha1` Kptfile is transformed to `info` 104 section in `v1` Kptfile. 105 2. `upstream` section, in the `v1alpha1` Kptfile is split into `upstream` and 106 `upstreamLock` sections in `v1` Kptfile. `upstream` section can be 107 modified by users to declare the desired version of the package, 108 `upstreamLock` is the resolved package state and should not be modified by 109 users. 110 3. The `OpenAPI` section in `v1alpha1` Kptfile is deprecated. 111 [Details below](#changes-to-setters). 112 4. `dependencies` section in `v1alpha1` Kptfile is deprecated. 113 [Details below](#sync-merged-with-update). 114 5. `functions` section in `v1alpha1` Kptfile holds definitions for Starlark 115 functions only. This section is deprecated and all the functions can be 116 declared in the `pipeline` section including [Starlark function]. 117 6. `inventory-object` is moved to `inventory` section in `v1` Kptfile. 118 [Details below](#live-group-changes). 119 120 ### `pkg` 121 122 #### `sync` merged with `update` 123 124 Nothing to worry about. `sync` is just a declarative version of `update` 125 functionality. The functionality is offered by `kpt pkg update` in a more user 126 friendly fashion. Running the new `kpt pkg update` command on a package which 127 has subpackages, simply means traversing the package hierarchy, and for each 128 package encountered, update the package if needed. If you want to declaratively 129 update nested subpackages, you can declare the desired version in the upstream 130 section of the respective subpackage and invoking `kpt pkg update` on the root 131 package will take care of updating all the nested subpackages. [Update guide] 132 133 ### `cfg` 134 135 #### Changes to Setters 136 137 `Setters` and `substitutions` no longer follow the complex and verbose OpenAPI 138 format. The setters and substitutions are simplified to use new and simple 139 setter pattern comments. Creating a setter is as simple as adding a line comment 140 to the desired field. Users need not add any `OpenAPI` definitions. Please refer 141 to [apply-setters] for the information about new syntax for setters and how to 142 apply setter values, this is equivalent to the `kpt cfg set` in `v0.39`. [Setter 143 Inheritance] works as usual in kpt `v1.0`. 144 145 Here is the [simple example] of setter syntax transformation. 146 147 #### Setter validations deprecated 148 149 We want to keep the authoring experience of setters as simple as possible and 150 hence [OpenAPI validations] and [required setters] feature offered by `v0.39` 151 setters is no longer offered in `v1.0` version of kpt. However, we are working 152 on providing an easy way to achieve similar functionality retaining the 153 simplicity of setters, which is scoped for post `v1.0` release. Stay tuned. 154 155 #### Auto setters deprecated 156 157 [Auto-setters] feature is deprecated in `v1.0` version of kpt. Since the setters 158 are migrated to a new and simple declarative version, package consumers can 159 easily declare all the setter values and render them all at once. 160 161 ### `fn` 162 163 #### `run` is split into `eval` and `render` 164 165 The functionality of `kpt fn run` command in `v0.39` is split into two different 166 CLI commands that execute functions corresponding to two fundamentally different 167 approaches: 168 169 `kpt fn render:` Executes the pipeline of functions declared in the package and 170 its subpackages. This is a declarative way to run functions. [render guide] 171 172 `kpt fn eval:` Executes a given function on the package. The image to run and 173 the functionConfig is specified as CLI argument. This is an imperative way to 174 run functions. [eval guide] 175 176 #### Function Config 177 178 As a result of these changes, we no longer need function configs to hold 179 `config.kubernetes.io/function` annotation. Functions can be declared in the 180 pipeline section of Kptfile and invoked using `kpt fn render`. The declared 181 function can point to the [function config] using `configPath` argument. 182 183 #### Function Results 184 185 In both `render` and `eval`, structured results can be enabled using the 186 `--results-dir` flag. Please refer to the [function results] section for more 187 information on the new structure of function results. 188 189 ### `live` 190 191 `kpt live` in `v1.0` no longer uses an inventory object to track the grouping of 192 resources in the cluster. Instead, it uses a more expressive `ResourceGroup` 193 CRD. Please refer to the user guide on [migrating inventory objects] to the 194 `ResourceGroup` equivalent. 195 196 ## Migration Steps 197 198 ### For Package Publishers 199 200 Based on the changes discussed above, this section walks package publishers through an end to 201 end workflow to migrate their existing packages which are compatible with 202 `v0.39` version of kpt, and make them compatible with kpt `v1.0`. 203 204 #### Automated portion of migration 205 206 Since you are the package publisher, you are expected to have the latest version 207 of published package on your local disk. If you do not already have it, you can [git clone] 208 the latest version of remote package on to your local disk. 209 210 ```shell 211 $ DEMO_HOME=$(mktemp -d); cd $DEMO_HOME 212 ``` 213 214 ```shell 215 # replace it with your package repo uri 216 $ git clone https://github.com/GoogleContainerTools/kpt-functions-catalog.git 217 ``` 218 219 ```shell 220 # cd to the package directory which you want to migrate 221 $ cd kpt-functions-catalog/testdata/fix/nginx-v1alpha1 222 ``` 223 224 ```shell 225 # verify the version of kpt 226 $ kpt version 227 1.0.0+ 228 ``` 229 230 Invoke `gcr.io/kpt-fn/fix` function on the kpt package. 231 232 ```shell 233 # you must be using 1.0+ version of kpt 234 $ kpt fn eval --image gcr.io/kpt-fn/fix:v0.2 --include-meta-resources --truncate-output=false 235 ``` 236 237 ```shell 238 # observe the changes done by the fix function 239 $ git diff 240 ``` 241 242 ##### Changes made by the function 243 244 1. Best effort is made by the function to transform the `packageMetaData` 245 section to the `info` section. 246 2. `upstream` section, in the `v1alpha1` Kptfile is converted to `upstream` and 247 `upstreamLock` sections in `v1` version of Kptfile. 248 3. `dependencies` section is removed from Kptfile. 249 4. Starlark functions section is removed from Kptfile. 250 5. `Setters` and `substitutions` are converted to new and simple setter 251 patterns. The setter comments in the resources are updated to follow new 252 comment syntax. 253 6. The `apply-setters` function and all the setters are added to the mutators 254 section in the pipeline. 255 7. Function annotation from function configs will be removed and corresponding 256 function definitions will be declared in `pipeline` section of Kptfile. 257 Reference to function config is added via [configPath] option. 258 259 Note: This function modifies only the local package files and doesn’t make any 260 changes to the resources in the live cluster. 261 262 #### Manual portion of migration 263 264 1. All the functions are treated as `mutators` by the `gcr.io/kpt-fn/fix` 265 function and are added to the `mutators` section in the pipeline. Users must 266 manually go through the functions and move the validator functions to the 267 `validators` section in the pipeline section of `v1` Kptfile. 268 1. The order of functions also must be re-arranged manually by users if 269 needed. 270 2. Also, note that the [function config] is used to configure the function 271 and is not included in the input list of resources to function. 272 2. [OpenAPI validations] and required setters feature offered by `v0.39` setters 273 is no longer offered in `v1.0` version of kpt. Users must write their own 274 validation functions to achieve the functionality. `Tip:` Adding a [starlark 275 function] would be an easier alternative to achieve the equivalent validation 276 functionality. 277 3. If you have used [Starlark runtime] in `v0.39`, please checkout the new and 278 improved [Starlark functions] and declare it accordingly in the pipeline. 279 4. [Auto-setters] feature is deprecated in `v1.0` version of kpt. Since the 280 setters are migrated to a new and simple declarative version, package 281 consumers can easily declare all the setter values and render them all at 282 once. 283 284 Test your migrated kpt package end-to-end and make sure that the 285 functionality is as expected. `gcr.io/kpt-fn/fix` is a helper for migration and 286 doesn't guarantee functional parity. 287 288 Finally, [publish your package] to git by upgrading the version so that your 289 consumers can fetch the specific version of the package. 290 291 ### For Package Consumers 292 293 This section walks package consumers through an end to end workflow in order to 294 fetch the latest version of the published package which is compatible with kpt `v1.0` 295 and migrate the local customizations(if any) already performed to their existing package. 296 297 - As a package consumer, you are expected to have some version of the kpt package 298 which is compatible with kpt `v0.39` on your local. 299 Fetch the latest version of published remote package in to a new directory different 300 from your existing package directory. 301 302 ```shell 303 $ DEMO_HOME=$(mktemp -d); cd $DEMO_HOME 304 ``` 305 306 ```shell 307 # verify the version of kpt 308 $ kpt version 309 1.0.0+ 310 ``` 311 312 ```shell 313 # fetch the package with upgraded version 314 $ kpt pkg get https://github.com/GoogleContainerTools/kpt-functions-catalog.git/testdata/fix/nginx-v1@master 315 ``` 316 317 - You might have performed some customizations to your existing package such as, 318 updated setter values, made some in-place edits to your resources etc. 319 Please make sure that you capture those customizations and add them to the newly 320 fetched package. 321 322 - Render the package resources with customizations 323 324 ```shell 325 $ kpt fn render nginx-v1/ 326 ``` 327 328 - The step is only applicable if you're using `kpt live` functionality. 329 a. If you are using the inventory object in order to manage live resources in the cluster, 330 please refer to `live migrate` command docs to perform [live migration]. 331 b. If you are using ResourceGroup CRD to manage live resources, copy the inventory 332 section in the Kptfile of existing package to the Kptfile of new package. 333 334 - Once you test your new package and confirm that all the changes are as expected, 335 you can simply discard the existing package and move forward with the new version 336 of the fetched and customized package. 337 338 Here is an [example kpt package], migrated from `v1alpha1` version(compatible with 339 kpt `v0.39`) to `v1` version(compatible with kpt `v1.0`). 340 341 ## Timeline 342 343 1. `Right now:` You can [install] and try the pre-release version of kpt `v1.0` 344 binary. 345 2. `June 1, 2021:` `gcr.io/kpt-fn/fix` function will be released in 346 [kpt-functions-catalog]. You can start migrating your existing kpt packages 347 using the function. 348 3. `July 1, 2021:` Package format `v1` will be released which guarantees 349 backwards compatability for new packages going forward. The existing kpt 350 packages are not compatible with the kpt `v1.0` binary and users will be 351 prompted to migrate/fix their packages. 352 4. `TBD:` Feature releases and bug fixes for pre-v1 versions of kpt will be 353 reserved for serious security bugs only. Users will be asked to migrate to 354 kpt `v1.0`. 355 356 [v0.39 commands]: https://googlecontainertools.github.io/kpt/reference/ 357 [v1.0 commands]: https://kpt.dev/reference/cli/ 358 [v1 kptfile]: https://github.com/GoogleContainerTools/kpt/blob/main/pkg/api/kptfile/v1/types.go 359 [starlark function]: https://catalog.kpt.dev/starlark/v0.2/ 360 [apply-setters]: https://catalog.kpt.dev/apply-setters/v0.1/ 361 [setter inheritance]: https://googlecontainertools.github.io/kpt/concepts/setters/#inherit-setter-values-from-parent-package 362 [openapi validations]: https://googlecontainertools.github.io/kpt/guides/producer/setters/#openapi-validations 363 [required setters]: https://googlecontainertools.github.io/kpt/guides/producer/setters/#required-setters 364 [auto-setters]: https://googlecontainertools.github.io/kpt/concepts/setters/#auto-setters 365 [migrating inventory objects]: https://googlecontainertools.github.io/kpt/reference/live/alpha/ 366 [live migration]: https://googlecontainertools.github.io/kpt/reference/cli/live/alpha/ 367 [configpath]: https://kpt.dev/book/04-using-functions/01-declarative-function-execution?id=configpath 368 [example kpt package]: https://github.com/GoogleContainerTools/kpt-functions-catalog/tree/master/testdata/fix 369 [simple example]: https://github.com/GoogleContainerTools/kpt-functions-catalog/tree/master/functions/go/fix#examples 370 [function config]: https://kpt.dev/book/04-using-functions/01-declarative-function-execution?id=configpath 371 [starlark runtime]: https://googlecontainertools.github.io/kpt/guides/producer/functions/starlark/ 372 [update guide]: https://kpt.dev/book/03-packages/05-updating-a-package 373 [render guide]: https://kpt.dev/book/04-using-functions/01-declarative-function-execution 374 [eval guide]: https://kpt.dev/book/04-using-functions/02-imperative-function-execution 375 [function results]: https://kpt.dev/book/04-using-functions/03-function-results 376 [the kpt book]: https://kpt.dev/book/ 377 [installation instructions]: https://kpt.dev/installation/ 378 [install]: https://kpt.dev/installation/ 379 [kpt-functions-catalog]: https://catalog.kpt.dev/ 380 [v1alpha1 kptfile]: https://github.com/GoogleContainerTools/kpt/blob/master/pkg/kptfile/pkgfile.go#L39 381 [git clone]: https://git-scm.com/docs/git-clone 382 [publish your package]: https://kpt.dev/book/03-packages/08-publishing-a-package