k8s.io/kubernetes@v1.31.0-alpha.0.0.20240520171757-56147500dadc/CHANGELOG/CHANGELOG-1.16.md (about) 1 <!-- BEGIN MUNGE: GENERATED_TOC --> 2 3 - [v1.16.15](#v11615) 4 - [Downloads for v1.16.15](#downloads-for-v11615) 5 - [Source Code](#source-code) 6 - [Client binaries](#client-binaries) 7 - [Server binaries](#server-binaries) 8 - [Node binaries](#node-binaries) 9 - [Changelog since v1.16.14](#changelog-since-v11614) 10 - [Changes by Kind](#changes-by-kind) 11 - [Bug or Regression](#bug-or-regression) 12 - [Dependencies](#dependencies) 13 - [Added](#added) 14 - [Changed](#changed) 15 - [Removed](#removed) 16 - [v1.16.14](#v11614) 17 - [Downloads for v1.16.14](#downloads-for-v11614) 18 - [Source Code](#source-code-1) 19 - [Client binaries](#client-binaries-1) 20 - [Server binaries](#server-binaries-1) 21 - [Node binaries](#node-binaries-1) 22 - [Changelog since v1.16.13](#changelog-since-v11613) 23 - [Changes by Kind](#changes-by-kind-1) 24 - [Bug or Regression](#bug-or-regression-1) 25 - [Other (Cleanup or Flake)](#other-cleanup-or-flake) 26 - [Dependencies](#dependencies-1) 27 - [Added](#added-1) 28 - [Changed](#changed-1) 29 - [Removed](#removed-1) 30 - [v1.16.13](#v11613) 31 - [Downloads for v1.16.13](#downloads-for-v11613) 32 - [Source Code](#source-code-2) 33 - [Client binaries](#client-binaries-2) 34 - [Server binaries](#server-binaries-2) 35 - [Node binaries](#node-binaries-2) 36 - [Changelog since v1.16.12](#changelog-since-v11612) 37 - [Urgent Upgrade Notes](#urgent-upgrade-notes) 38 - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade) 39 - [Changes by Kind](#changes-by-kind-2) 40 - [Bug or Regression](#bug-or-regression-2) 41 - [Dependencies](#dependencies-2) 42 - [Added](#added-2) 43 - [Changed](#changed-2) 44 - [Removed](#removed-2) 45 - [v1.16.12](#v11612) 46 - [Downloads for v1.16.12](#downloads-for-v11612) 47 - [Source Code](#source-code-3) 48 - [Client binaries](#client-binaries-3) 49 - [Server binaries](#server-binaries-3) 50 - [Node binaries](#node-binaries-3) 51 - [Changelog since v1.16.11](#changelog-since-v11611) 52 - [Changes by Kind](#changes-by-kind-3) 53 - [API Change](#api-change) 54 - [Bug or Regression](#bug-or-regression-3) 55 - [Dependencies](#dependencies-3) 56 - [Added](#added-3) 57 - [Changed](#changed-3) 58 - [Removed](#removed-3) 59 - [v1.16.12-rc.1](#v11612-rc1) 60 - [Downloads for v1.16.12-rc.1](#downloads-for-v11612-rc1) 61 - [Source Code](#source-code-4) 62 - [Client binaries](#client-binaries-4) 63 - [Server binaries](#server-binaries-4) 64 - [Node binaries](#node-binaries-4) 65 - [Changelog since v1.16.11](#changelog-since-v11611-1) 66 - [Changes by Kind](#changes-by-kind-4) 67 - [API Change](#api-change-1) 68 - [Bug or Regression](#bug-or-regression-4) 69 - [Dependencies](#dependencies-4) 70 - [Added](#added-4) 71 - [Changed](#changed-4) 72 - [Removed](#removed-4) 73 - [v1.16.11](#v11611) 74 - [Downloads for v1.16.11](#downloads-for-v11611) 75 - [Source Code](#source-code-5) 76 - [Client binaries](#client-binaries-5) 77 - [Server binaries](#server-binaries-5) 78 - [Node binaries](#node-binaries-5) 79 - [Changelog since v1.16.10](#changelog-since-v11610) 80 - [Changes by Kind](#changes-by-kind-5) 81 - [API Change](#api-change-2) 82 - [Feature](#feature) 83 - [Bug or Regression](#bug-or-regression-5) 84 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-1) 85 - [Dependencies](#dependencies-5) 86 - [Added](#added-5) 87 - [Changed](#changed-5) 88 - [Removed](#removed-5) 89 - [v1.16.10](#v11610) 90 - [Downloads for v1.16.10](#downloads-for-v11610) 91 - [Source Code](#source-code-6) 92 - [Client binaries](#client-binaries-6) 93 - [Server binaries](#server-binaries-6) 94 - [Node binaries](#node-binaries-6) 95 - [Changelog since v1.16.9](#changelog-since-v1169) 96 - [Changes by Kind](#changes-by-kind-6) 97 - [API Change](#api-change-3) 98 - [Bug or Regression](#bug-or-regression-6) 99 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-2) 100 - [Dependencies](#dependencies-6) 101 - [Added](#added-6) 102 - [Changed](#changed-6) 103 - [Removed](#removed-6) 104 - [v1.16.9](#v1169) 105 - [Downloads for v1.16.9](#downloads-for-v1169) 106 - [Client Binaries](#client-binaries-7) 107 - [Server Binaries](#server-binaries-7) 108 - [Node Binaries](#node-binaries-7) 109 - [Changelog since v1.16.8](#changelog-since-v1168) 110 - [Changes by Kind](#changes-by-kind-7) 111 - [Feature](#feature-1) 112 - [Bug or Regression](#bug-or-regression-7) 113 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-3) 114 - [v1.16.8](#v1168) 115 - [Downloads for v1.16.8](#downloads-for-v1168) 116 - [Client Binaries](#client-binaries-8) 117 - [Server Binaries](#server-binaries-8) 118 - [Node Binaries](#node-binaries-8) 119 - [Changelog since v1.16.7](#changelog-since-v1167) 120 - [Changes by Kind](#changes-by-kind-8) 121 - [API Change](#api-change-4) 122 - [Other (Bug, Cleanup or Flake)](#other-bug-cleanup-or-flake) 123 - [v1.16.7](#v1167) 124 - [Downloads for v1.16.7](#downloads-for-v1167) 125 - [Client Binaries](#client-binaries-9) 126 - [Server Binaries](#server-binaries-9) 127 - [Node Binaries](#node-binaries-9) 128 - [Changelog since v1.16.6](#changelog-since-v1166) 129 - [Changes by Kind](#changes-by-kind-9) 130 - [Other (Bug, Cleanup or Flake)](#other-bug-cleanup-or-flake-1) 131 - [v1.16.6](#v1166) 132 - [Downloads for v1.16.6](#downloads-for-v1166) 133 - [Client Binaries](#client-binaries-10) 134 - [Server Binaries](#server-binaries-10) 135 - [Node Binaries](#node-binaries-10) 136 - [Changelog since v1.16.5](#changelog-since-v1165) 137 - [v1.16.5](#v1165) 138 - [Downloads for v1.16.5](#downloads-for-v1165) 139 - [Client Binaries](#client-binaries-11) 140 - [Server Binaries](#server-binaries-11) 141 - [Node Binaries](#node-binaries-11) 142 - [Changelog since v1.16.4](#changelog-since-v1164) 143 - [Other notable changes](#other-notable-changes) 144 - [v1.16.4](#v1164) 145 - [Downloads for v1.16.4](#downloads-for-v1164) 146 - [Client Binaries](#client-binaries-12) 147 - [Server Binaries](#server-binaries-12) 148 - [Node Binaries](#node-binaries-12) 149 - [Changelog since v1.16.3](#changelog-since-v1163) 150 - [Other notable changes](#other-notable-changes-1) 151 - [v1.16.3](#v1163) 152 - [Downloads for v1.16.3](#downloads-for-v1163) 153 - [Client Binaries](#client-binaries-13) 154 - [Server Binaries](#server-binaries-13) 155 - [Node Binaries](#node-binaries-13) 156 - [Changelog since v1.16.2](#changelog-since-v1162) 157 - [Other notable changes](#other-notable-changes-2) 158 - [v1.16.2](#v1162) 159 - [Downloads for v1.16.2](#downloads-for-v1162) 160 - [Client Binaries](#client-binaries-14) 161 - [Server Binaries](#server-binaries-14) 162 - [Node Binaries](#node-binaries-14) 163 - [Changelog since v1.16.1](#changelog-since-v1161) 164 - [Other notable changes](#other-notable-changes-3) 165 - [v1.16.1](#v1161) 166 - [Downloads for v1.16.1](#downloads-for-v1161) 167 - [Client Binaries](#client-binaries-15) 168 - [Server Binaries](#server-binaries-15) 169 - [Node Binaries](#node-binaries-15) 170 - [Changelog since v1.16.0](#changelog-since-v1160) 171 - [Other notable changes](#other-notable-changes-4) 172 - [v1.16.0](#v1160) 173 - [Downloads for v1.16.0](#downloads-for-v1160) 174 - [Client Binaries](#client-binaries-16) 175 - [Server Binaries](#server-binaries-16) 176 - [Node Binaries](#node-binaries-16) 177 - [Kubernetes v1.16.0 Release Notes](#kubernetes-v1160-release-notes) 178 - [What’s New (Major Themes)](#what’s-new-major-themes) 179 - [Additional Notable Feature Updates](#additional-notable-feature-updates) 180 - [Known Issues](#known-issues) 181 - [Urgent Upgrade Notes](#urgent-upgrade-notes-1) 182 - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade-1) 183 - [Cluster Lifecycle](#cluster-lifecycle) 184 - [Storage](#storage) 185 - [Deprecations and Removals](#deprecations-and-removals) 186 - [Metrics Changes](#metrics-changes) 187 - [Added metrics](#added-metrics) 188 - [Removed metrics](#removed-metrics) 189 - [Deprecated/changed metrics](#deprecated/changed-metrics) 190 - [Notable Features](#notable-features) 191 - [Beta](#beta) 192 - [Alpha](#alpha) 193 - [CLI Improvements](#cli-improvements) 194 - [Misc](#misc) 195 - [API Changes](#api-changes) 196 - [Other notable changes](#other-notable-changes-5) 197 - [API Machinery](#api-machinery) 198 - [Apps](#apps) 199 - [Auth](#auth) 200 - [CLI](#cli) 201 - [Cloud Provider](#cloud-provider) 202 - [Cluster Lifecycle](#cluster-lifecycle-1) 203 - [Instrumentation](#instrumentation) 204 - [Network](#network) 205 - [Node](#node) 206 - [Scheduling](#scheduling) 207 - [Storage](#storage-1) 208 - [Testing](#testing) 209 - [Windows](#windows) 210 - [Dependencies](#dependencies-7) 211 - [Changed](#changed-7) 212 - [Unchanged](#unchanged) 213 - [Removed](#removed-7) 214 - [Detailed go Dependency Changes](#detailed-go-dependency-changes) 215 - [Added](#added-7) 216 - [Changed](#changed-8) 217 - [Removed](#removed-8) 218 - [v1.16.0-rc.2](#v1160-rc2) 219 - [Downloads for v1.16.0-rc.2](#downloads-for-v1160-rc2) 220 - [Client Binaries](#client-binaries-17) 221 - [Server Binaries](#server-binaries-17) 222 - [Node Binaries](#node-binaries-17) 223 - [Changelog since v1.16.0-rc.1](#changelog-since-v1160-rc1) 224 - [Other notable changes](#other-notable-changes-6) 225 - [v1.16.0-rc.1](#v1160-rc1) 226 - [Downloads for v1.16.0-rc.1](#downloads-for-v1160-rc1) 227 - [Client Binaries](#client-binaries-18) 228 - [Server Binaries](#server-binaries-18) 229 - [Node Binaries](#node-binaries-18) 230 - [Changelog since v1.16.0-beta.2](#changelog-since-v1160-beta2) 231 - [Other notable changes](#other-notable-changes-7) 232 - [v1.16.0-beta.2](#v1160-beta2) 233 - [Downloads for v1.16.0-beta.2](#downloads-for-v1160-beta2) 234 - [Client Binaries](#client-binaries-19) 235 - [Server Binaries](#server-binaries-19) 236 - [Node Binaries](#node-binaries-19) 237 - [Changelog since v1.16.0-beta.1](#changelog-since-v1160-beta1) 238 - [Other notable changes](#other-notable-changes-8) 239 - [v1.16.0-beta.1](#v1160-beta1) 240 - [Downloads for v1.16.0-beta.1](#downloads-for-v1160-beta1) 241 - [Client Binaries](#client-binaries-20) 242 - [Server Binaries](#server-binaries-20) 243 - [Node Binaries](#node-binaries-20) 244 - [Changelog since v1.16.0-alpha.3](#changelog-since-v1160-alpha3) 245 - [Action Required](#action-required) 246 - [Other notable changes](#other-notable-changes-9) 247 - [v1.16.0-alpha.3](#v1160-alpha3) 248 - [Downloads for v1.16.0-alpha.3](#downloads-for-v1160-alpha3) 249 - [Client Binaries](#client-binaries-21) 250 - [Server Binaries](#server-binaries-21) 251 - [Node Binaries](#node-binaries-21) 252 - [Changelog since v1.16.0-alpha.2](#changelog-since-v1160-alpha2) 253 - [Action Required](#action-required-1) 254 - [Other notable changes](#other-notable-changes-10) 255 - [v1.16.0-alpha.2](#v1160-alpha2) 256 - [Downloads for v1.16.0-alpha.2](#downloads-for-v1160-alpha2) 257 - [Client Binaries](#client-binaries-22) 258 - [Server Binaries](#server-binaries-22) 259 - [Node Binaries](#node-binaries-22) 260 - [Changelog since v1.16.0-alpha.1](#changelog-since-v1160-alpha1) 261 - [Action Required](#action-required-2) 262 - [Other notable changes](#other-notable-changes-11) 263 - [v1.16.0-alpha.1](#v1160-alpha1) 264 - [Downloads for v1.16.0-alpha.1](#downloads-for-v1160-alpha1) 265 - [Client Binaries](#client-binaries-23) 266 - [Server Binaries](#server-binaries-23) 267 - [Node Binaries](#node-binaries-23) 268 - [Changelog since v1.15.0](#changelog-since-v1150) 269 - [Action Required](#action-required-3) 270 - [Other notable changes](#other-notable-changes-12) 271 272 <!-- END MUNGE: GENERATED_TOC --> 273 274 # v1.16.15 275 276 277 ## Downloads for v1.16.15 278 279 ### Source Code 280 281 filename | sha512 hash 282 -------- | ----------- 283 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes.tar.gz) | 795a26268a5e553eb39157c2d46e8cd1855b4962a0004524286883427124807323c3db5d6258536fcf4c3a0e1b91636662c6010a04020a37105db3f0ba0cca4e 284 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-src.tar.gz) | e7b9360da8a3f65d5df78f0af7c9209665da3382aba79d7f26c8771a8658effffbf9ed0ee36b87eb409c68a7b910baade351b6cb59d129c163f900e9e843cf2f 285 286 ### Client binaries 287 288 filename | sha512 hash 289 -------- | ----------- 290 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-darwin-386.tar.gz) | 8090762a85160a27f22073f436f330fa36a606dd6c25b8350ab0d2e1f5b785b583697b700fd4f7708eb53ac3e087e3b33436fdf49ee13c455a2f2f029d3cafc0 291 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-darwin-amd64.tar.gz) | e1f899a90011f83df8773cfda36b89aee2e68dc09e06119727c954a520623a593f769821ca547dd5e0335f95cdcce28cd579aa47abced9771f52e2cee036b5b2 292 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-linux-386.tar.gz) | 4e93e9a8831b175da62f42856970abe177c44bf637997ce03e8d1497115cc3b9e336a53bd2758309999b68b25111f05bd9ed0d247857a037b8d16f136e2112b3 293 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-linux-amd64.tar.gz) | f9b2af052eccb63b3c8f94144704713e9a9b385eda0b2e5ae4ecf5592834fbe80f891e876a31bed80115cc4a225346d8815dd66280b38b314d639eaa66d7fcaf 294 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-linux-arm.tar.gz) | b77d6bdbefe1dbf41d07b3287d68659c42c3f3297614d7b0d1f591f2c5fa552fa5390fe6ab2759b6fbae17b8c60fee93e56a7ecd292d7332daedb65710fac7fd 295 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-linux-arm64.tar.gz) | 7b77f0d1455369d436460a1b534756de36aa2f1ca93e6cd26f0f2f6261f28d8dd0e0c214a115cca63a2270d8d0bce9b4c2606357405c07d4bc5647cfd6d33470 296 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-linux-ppc64le.tar.gz) | 9500f8988fe1553a80f6d5c14bf5622a0f7fcd587feb7156578b92fca8f930a914027ebad961244e16fa892f916f0b3e1f7fae2bd9791173eb610bb44cfa6eae 297 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-linux-s390x.tar.gz) | 7c19066d069018615ed79a85b651b4af93d810937cc50f2a5a18361c0cde60d112ed9bfd87645eec4d784e85c480dd384d91e30b59bd1a53b59ba7be91b08dd0 298 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-windows-386.tar.gz) | 91c5813fa83adb4bc1274fde4437d203882a65f800b1daf178caceb8f2ba57fc6f55e9d0d1bc2f6f6fb313bc2ac2a1a9e2bb5b956a2bd01556ba8e67feeb711e 299 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-client-windows-amd64.tar.gz) | 4fe480001c6ecafb2df73cfe628edaab718b5584e8856da8a9da28c4d83c1e6ffa8f96cee51309e915926e5e22d7f13b7ef8a7d5260f11e3d13f655788fe5fee 300 301 ### Server binaries 302 303 filename | sha512 hash 304 -------- | ----------- 305 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-server-linux-amd64.tar.gz) | f987b88ecb7d3b83b18f060703596b9e2dd5098057c2542b16debfc8385981402a34436c65e306048c7ce60409515f5b40e5eaab8b55c8c619923f9ed528e8a4 306 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-server-linux-arm.tar.gz) | 168c11047482a3ca3a06d94db269ddc3320f68372e787588ff1793f519d8d945c56b96e713b50fffb492a0eadb4c27b67c75f97c784a85501f4f2aa8a6ed21d0 307 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-server-linux-arm64.tar.gz) | 71d9cf683f194915a9d4b12ee84d8218232dac8ae5b4b3a0992d7702ce7641bda49c528dddb2b551788b69e5d27783a38f781e9aee2fa0d799571a34ca4fe3aa 308 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-server-linux-ppc64le.tar.gz) | 512256b1cea99a92d9c00de31fbde93d34979c10646f38a853d7c9481ab71e1439b22780a86fea41b8424ef45ff2bdf32afab001c3353e223ffe37d642c34df0 309 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-server-linux-s390x.tar.gz) | f0c757c1b687fa21e510ab720d793a75c4dc4ba7885f44f56a60960e9deb7fca62abb81d8da6259a32086204ab6a3f1e397c535a145568e156577e7dac62a2d7 310 311 ### Node binaries 312 313 filename | sha512 hash 314 -------- | ----------- 315 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-node-linux-amd64.tar.gz) | 7a8bd0a08a786da788f602ef977b0da5a1556456ed0979e15c51916b4dc618d31024c272320a5955e89711651dda2fcce6f21c993542f5fcbb0c4aa2bff0c094 316 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-node-linux-arm.tar.gz) | 18ab5567ef9812f8ab8efdae9df5e158e65eba2a18bd7f1d10b79027517a44b1654c97320108637e0e9af7fd155b3124c5c35eedccbec334621c71b3c30699f2 317 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-node-linux-arm64.tar.gz) | d3c339cae2ae454a21f24a2e1d9c935d3c87044ca38d1062e223c4ec2184c7e55392b4764d3bb46dc7375abca84ccc0d1f9c1cbc2af8848bb950ec0ac2d60150 318 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-node-linux-ppc64le.tar.gz) | d613bafa6f6678f58874b6595a6b3ada17e83ce09bc1da4e1c018c9909ecf4f3ae462396c25eeeb5ca29e0b44b0c67cf5a1347df20dde54900058f750882ea26 319 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-node-linux-s390x.tar.gz) | 72a5b835868776f9d8d561c401d53e2f059f14328fce101fa28a8948e72b51853beabcb45a9808e8660a5ddf7c19b4564357aeb740ed22d3fae3acc9e69e8d2e 320 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.15/kubernetes-node-windows-amd64.tar.gz) | 8297003cf6036ac1ceee670f2ad38822118dbaf89a2acbae705c23ae8f03f788184c761802956c1b7905d90bcffcd76e160709fb2acffffa5d163939048480bb 321 322 ## Changelog since v1.16.14 323 324 ## Changes by Kind 325 326 ### Bug or Regression 327 328 - Correctly handle resetting cpuacct in a live container ([#94040](https://github.com/kubernetes/kubernetes/pull/94040), [@andyzhangx](https://github.com/andyzhangx)) [SIG Node and Windows] 329 - Fix uint64 overflow when elapsed ([#94038](https://github.com/kubernetes/kubernetes/pull/94038), [@andyzhangx](https://github.com/andyzhangx)) [SIG Node and Windows] 330 - Fix: incorrect max azure disk max count ([#92331](https://github.com/kubernetes/kubernetes/pull/92331), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 331 - Fixes a bug evicting pods after a taint with a limited tolerationSeconds toleration is removed from a node ([#93722](https://github.com/kubernetes/kubernetes/pull/93722), [@liggitt](https://github.com/liggitt)) [SIG Apps and Node] 332 - Fixes an issue that can result in namespaced custom resources being orphaned when their namespace is deleted, if the CRD defining the custom resource is removed concurrently with namespaces being deleted, then recreated. ([#93790](https://github.com/kubernetes/kubernetes/pull/93790), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Apps] 333 - Kube-apiserver: fixed a bug returning inconsistent results from list requests which set a field or label selector and set a paging limit ([#94002](https://github.com/kubernetes/kubernetes/pull/94002), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery] 334 - Use NLB Subnet CIDRs instead of VPC CIDRs in Health Check SG Rules ([#93515](https://github.com/kubernetes/kubernetes/pull/93515), [@t0rr3sp3dr0](https://github.com/t0rr3sp3dr0)) [SIG Cloud Provider] 335 336 ## Dependencies 337 338 ### Added 339 _Nothing has changed._ 340 341 ### Changed 342 - github.com/evanphx/json-patch: [162e562 → v4.9.0+incompatible](https://github.com/evanphx/json-patch/compare/162e562...v4.9.0) 343 344 ### Removed 345 - github.com/jessevdk/go-flags: [v1.4.0](https://github.com/jessevdk/go-flags/tree/v1.4.0) 346 347 348 349 # v1.16.14 350 351 352 ## Downloads for v1.16.14 353 354 ### Source Code 355 356 filename | sha512 hash 357 -------- | ----------- 358 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes.tar.gz) | 6c55864d3666aadcbec712e7cc68d6c3e2f7efd7ce3880250c56bc237506fc88f94fa778da466fb4efe724bf5b216dc9a3e7f5e576574b6d9a8745ce40cf3c37 359 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-src.tar.gz) | ae1b2951e5fbaa6841d1d98d472c1d0f0652edebfa941f18d144124040a0281f0d70eef0b66324d35d7e08fdb8d20478591118a437a240b9cae4d669b37f1b76 360 361 ### Client binaries 362 363 filename | sha512 hash 364 -------- | ----------- 365 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-darwin-386.tar.gz) | a9541c9078639169ec4c35af238447dca1bdb9200f6f2515d9e10d5395ab78aa4f52681849d6945e2bc42f4aeae22a9d25592ef5df509250bccc4cc655776254 366 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-darwin-amd64.tar.gz) | 41e597fcb6d0f4ed2dfdd0ce1b3610375f80d9f196de7f6ff202eb4afb694fcf0b8e7a787ba308a953d48323883e5791137535626d9169e88a5c1a56f99514a2 367 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-linux-386.tar.gz) | 29ff8f0c5aa363594d4dfd9bde811c0b446c3a4d0b2084685ec7c63af32a1c9066a5e7f3be959d0914288d9dfe9f171cf3d6e2db3511bc15abb6543dd1508a4d 368 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-linux-amd64.tar.gz) | 86d20c7c8d6bc61a82aa47aea136160867978df1bd04a6a23d46c04b814b52a0335f81f9c44882db2c12c551e84ea34bd2fc9934daa64ad46929a3082faeb32f 369 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-linux-arm.tar.gz) | 75b6d341a18bb2d5cc48957bac738151047904ccbb001450a18fd705a3064963d6b8df08bc099d683d383af365743f884fbfd2f97160065cb0ab51693efd94db 370 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-linux-arm64.tar.gz) | 5513b6e7e598906f6f7c391516d68f06686cf644d19eb31110c194c53fbf4465635c578636e7f876eb4aa99141c7463b131cf8da84c3485b3928601cd06c7d64 371 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-linux-ppc64le.tar.gz) | 7ad76ce53abee3f1381cc0cb69e4183dd265744cf608f465ff2f18266edbb8ad72e4ba0fda1794f59cce2c3764bb61bf65537dbc9fb7e6fbb2d1ed6518c6ad40 372 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-linux-s390x.tar.gz) | a6f7ee7bcfd48fc56e33323a05670a27fdb58ca965660a94e91af99dff25b7fbe73041fdb98e61966351dcbabc693f61b603bdf1b200e3dd20ce4e13d3ccb8f2 373 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-windows-386.tar.gz) | ffce29187ed9721709cb0f53dd95ab92973a877e1af6874186f61fc610d710afd7d213cebd57177ce729f2e045262ab9383a010b823e188cb0d685bc06ae8072 374 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-client-windows-amd64.tar.gz) | ebafa35cc5cc07a43dcb52e36475ae55c9ed505728eae12d470d2f05a26c4b4e7001377d5057952ed2e482ad01079f7b9efb4f9d489511c8f31277a24df251b2 375 376 ### Server binaries 377 378 filename | sha512 hash 379 -------- | ----------- 380 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-server-linux-amd64.tar.gz) | 12698c56c7dd1f87a24a1b49969b74940246a24aadc7dc2f96f175bb1b8293632dbd2b78da7e5490574778c5b045821105623e1273640f3eec1156cdfdd8c046 381 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-server-linux-arm.tar.gz) | 89d573c24ffc039c89bee51144fea17bf363f46bc9b2c575b6ed2cd7422cea79a16d6f0ccbbc68692cd754e4a7c6f8b9fd7c5282e6e0e714203b2de34d4e4979 382 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-server-linux-arm64.tar.gz) | af7cfd6d1fc7b8eef636acfc3b35411d63607a57d1273e37b724f01a9e9f6afea7ad3366689d3a80ca31b6f8c0dc4db5aa91e94984967c3ba20002c1f50caa2d 383 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-server-linux-ppc64le.tar.gz) | e5b27fb3993041f65d6e954e23d7918746229fd39ee3a69ba834e7d8e4f04a669eb075d98de86df7056cf50a25cc4f3baae2f4d97e16398bcfad6239fa37fd12 384 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-server-linux-s390x.tar.gz) | f6b6b7457a31e3d4cf19762791966d477e845ab37a94ea8ce037c949072dc90ca7fe54be7aac9d0bbceda5ec518d97a2c8aca4af1434b26bd8ba7190a2d2656d 385 386 ### Node binaries 387 388 filename | sha512 hash 389 -------- | ----------- 390 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-node-linux-amd64.tar.gz) | 0d5184716900e8263c99ebfba2cc50b20156dc6c01cafabc1aec2815a89187b534ed167ab202e12d5f5aaf8ff15396737a0b62aa3437c0812d91173a5893ac5f 391 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-node-linux-arm.tar.gz) | 5a8c336c629a4afdc5bc3ac29c1d0feb1afd742d0cfa4c3aea23421fa7d4020b9926b6f077777f4b9a298e868e86ba745f70e43e0dd16b2ac000225e4670fdf1 392 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-node-linux-arm64.tar.gz) | 7a3925fa1be72dbcda446590484a867c85e3828890a8dfdfbca223a5db9395ba38763f88d8601a34b1d621f2af0d799d41904f51e43efe135b01f54daef31c78 393 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-node-linux-ppc64le.tar.gz) | 694a9b50751cb09d7f46b97e8d5a99a3d552265e8b7c0761272d987c37a6e911008840a472752063f18b47575080c5fc110f1cbb26b9c2c6ce72de5b4fd421ce 394 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-node-linux-s390x.tar.gz) | 942d2ab526fe76851b923f4f2d5c4a4f2af12325e4a3013180797218e55dc078684f3319a75944824a16f53ea1e2f017bc59b0c41e063ffcf72f2dec806a1ab4 395 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.14/kubernetes-node-windows-amd64.tar.gz) | bdb1fb0eeed3170ff2764aea27fe695d4d4eb313371033f633c06ac5666de668a9c16704666c97548fc8e67a2774652b660dcdfa12a861d833056021234d8a92 396 397 ## Changelog since v1.16.13 398 399 ## Changes by Kind 400 401 ### Bug or Regression 402 403 - Do not add nodes labeled with kubernetes.azure.com/managed=false to backend pool of load balancer. ([#93034](https://github.com/kubernetes/kubernetes/pull/93034), [@matthias50](https://github.com/matthias50)) [SIG Cloud Provider] 404 - Fix instance not found issues when an Azure Node is recreated in a short time ([#93316](https://github.com/kubernetes/kubernetes/pull/93316), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 405 - Fix: don't use docker config cache if it's empty ([#92330](https://github.com/kubernetes/kubernetes/pull/92330), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 406 - Fix: initial delay in mounting azure disk & file ([#93052](https://github.com/kubernetes/kubernetes/pull/93052), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 407 - Fixed a performance issue applying json patches to deeply nested objects ([#93813](https://github.com/kubernetes/kubernetes/pull/93813), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, CLI, Cloud Provider and Cluster Lifecycle] 408 - Fixed a regression in kubeadm manifests for kube-scheduler and kube-controller-manager which caused continuous restarts because of failing health checks ([#93208](https://github.com/kubernetes/kubernetes/pull/93208), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] 409 - Fixes a regression in kube-apiserver causing 500 errors from the `/readyz` endpoint ([#93645](https://github.com/kubernetes/kubernetes/pull/93645), [@ialidzhikov](https://github.com/ialidzhikov)) [SIG API Machinery] 410 411 ### Other (Cleanup or Flake) 412 413 - Build: Update Debian base images 414 - debian-base:v2.1.3 415 - debian-iptables:v12.1.2 416 - debian-hyperkube-base:v1.1.3 ([#93927](https://github.com/kubernetes/kubernetes/pull/93927), [@justaugustus](https://github.com/justaugustus)) [SIG API Machinery, Cluster Lifecycle and Release] 417 - Kubernetes is now built with go1.13.15 ([#93956](https://github.com/kubernetes/kubernetes/pull/93956), [@justaugustus](https://github.com/justaugustus)) [SIG Release and Testing] 418 - Update Golang to v1.13.14 419 - Update bazel to 2.2.0 420 - Update repo-infra to 0.0.8 (to support go1.14.6 and go1.13.14) 421 - Includes: 422 - bazelbuild/bazel-toolchains@3.4.0 423 - bazelbuild/rules_go@v0.22.8 ([#93235](https://github.com/kubernetes/kubernetes/pull/93235), [@justaugustus](https://github.com/justaugustus)) [SIG API Machinery, Release and Testing] 424 425 ## Dependencies 426 427 ### Added 428 - github.com/jessevdk/go-flags: [v1.4.0](https://github.com/jessevdk/go-flags/tree/v1.4.0) 429 430 ### Changed 431 - github.com/evanphx/json-patch: [v4.2.0+incompatible → 162e562](https://github.com/evanphx/json-patch/compare/v4.2.0...162e562) 432 433 ### Removed 434 _Nothing has changed._ 435 436 437 438 # v1.16.13 439 440 441 ## Downloads for v1.16.13 442 443 ### Source Code 444 445 filename | sha512 hash 446 -------- | ----------- 447 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes.tar.gz) | 8491ac95464beb9333e3bc731bfe024af63af568e8fe13ad3ea727338167ec58ee361896e26d4bd0688317f243876b531b07409ef229bbdd659a69093d7c4e7d 448 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-src.tar.gz) | edaca169bcb59b42a165b9343337c0c574fbcfc97291b84a7bd50d38afa468791cde4db983ae9019a6ba0d10e5b0aeba3665690c54139d21a7711e0118d2c517 449 450 ### Client binaries 451 452 filename | sha512 hash 453 -------- | ----------- 454 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-darwin-386.tar.gz) | 6e720c2259ec62f0d898804ca55c20ea50034c15a5871b7037edd11c043f95243c792a9fb1890084351ddf3bb7d8ad9e549d80b817f904224033183ddd5ee47b 455 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-darwin-amd64.tar.gz) | 0fe4ac6853922b7d20b5381991dda48eb9ad11c79b368516e3c1c83227dc617f540f64040550f60bdc2892fc6fda6abc13464a9e51376ccb4ac860fb0154bffb 456 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-linux-386.tar.gz) | 2d3f59f907311e0ccbc261ac9f0ad24876f6954f2e22d4954456f6c7a1ea99616d97beecf352fc90b225d1ec20df45e599ed0dc9e90f46ac56c843346766fb4f 457 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-linux-amd64.tar.gz) | 587c9f192fbc46e04201906fd8e24c71b18838c13722b743b1d7b60d65cd0c1bd48ffa4bd7e82a9b3f05ff482747f2afde17975e01d2400b14ce3ed864f4308d 458 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-linux-arm.tar.gz) | 757f8b512f28d4c62b217dec13963f21bb0dc6ec083deef74974c79f2d989c033a88a8a5d7c206304d2f7499da7343dbc2015f4000e8d38c83fa524283e09721 459 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-linux-arm64.tar.gz) | 6ec7749f380185f8db968964af9822debc509dd4168242a142bbbcd1f2bfb3310534b985c6280f18684e1670d7fbb82806c471a1d87e44c08a13836915f6e9ae 460 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-linux-ppc64le.tar.gz) | e34fe6eeb48e40755201e72270077cb7e78e9c9832cae6e6ba59c643f47c2079310577357563908ad79e9d46704eb1abdae93b2671bc7b5afeb0964f4e47f4b5 461 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-linux-s390x.tar.gz) | 1efb4e00b80c6d59472d94e2a016f1e4e48025e7d204a93a89796574c28a6705386fc4eea8a3364083de2b74a0378a61c392b45a975c605f56b1c5192b232a0e 462 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-windows-386.tar.gz) | ad450577697549e8a492edf4a2cb895c854aa726c3d1025782c1834308619f6123cee1d7072152d7bd5d5e707aa8800e1a88a1a1738813f7aeb9b67d8e12cd1a 463 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-client-windows-amd64.tar.gz) | 151854b9ad1ddda0f6027248ef369b55f05cc7bb301a2a9def40f4acb62b09612623bf1f862b6ff2f4d51abf12352877c79bbaa6dc37d891ffdab461b70f1441 464 465 ### Server binaries 466 467 filename | sha512 hash 468 -------- | ----------- 469 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-server-linux-amd64.tar.gz) | 59b9cc215e8cae581c0918fcdfc5bde2dd7ed871e5a71ee9b17fccc2d951e20e74e05abf0091ebd52801b8c998df560860844ab9b1b00d3a76ae3919ca0a338f 470 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-server-linux-arm.tar.gz) | 5d1a9266d1a741933c7e7d5cda3ccefd064d299a35b282df04266aeeab3744ffb160f754c1098413341743ac280405d30e57d2b4a70deb32251a21d79b6f640f 471 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-server-linux-arm64.tar.gz) | a0bc61b3ae50fd039388caf23d959666169f4f4537db4a835ad6af2035113c36e7249058242b4090ea89f86671d1def8d8a555b2680741a4b741e2b0b2e6a119 472 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-server-linux-ppc64le.tar.gz) | 1636bb2837ec997659648f33505cbaf32e567d6073e6d636180d7a3e603762a8df38d925fe3b7978e218d6e6dc168e884732a4b867dc0b0e60dd3558ab71b5ae 473 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-server-linux-s390x.tar.gz) | 10db780e2abe2031888ee48c16ea023ffb799937c50c3505357083d7ce2adfc302975d3af1a50312ffd058b3f6902bc0e4c6f47e7be537ccfbbb539f58f36897 474 475 ### Node binaries 476 477 filename | sha512 hash 478 -------- | ----------- 479 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-node-linux-amd64.tar.gz) | fb2381e684b3260a4d5a3b6410054bc9521f545b3730fd76de8131c6625019290d43fe5d611b774e8f3ff1ed0f2044e6a7104fc2aa30ef9fffa20383d958e415 480 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-node-linux-arm.tar.gz) | 537cd5bcf614cef80409962294b34537e7977224e82aadb66706f3a6c6fcb3de8cc696c7ddea3b04838f006d2a89d289dc38a92e7fd2c7abb8ca77a0b53e4b7f 481 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-node-linux-arm64.tar.gz) | 07f38550fecc6e9ecbcbbdf74db6ca4ba71b4081d6d7077db9c583da56e6d56ce5bd84211f1f2c3657d8b0dc52c1c533e33e8e174443fd5f170c4168485a0bde 482 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-node-linux-ppc64le.tar.gz) | 34416a3dc2b1e6c38b658c565539246830dec46bb6a0daf093eab0897c82fe0f364102537783dfd802443ddb637df67c13b2fa72035b44a05e2b73b96a3ad594 483 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-node-linux-s390x.tar.gz) | fdc87ad8aec7cffbf0f95d0bdec18325e0966b3b935ec288d2d14d612d531e12fc8d2085a3afe81a97b81440fd782ccc898d4d05ec531bee9e7159c168d5469c 484 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.13/kubernetes-node-windows-amd64.tar.gz) | abfaa76e0c67067668013ebba39c70a76dff291da2da403d1161785979051c09e6f40d59513ae76c5248c3e775c4484219cad72783adf0204731051f09f87daa 485 486 ## Changelog since v1.16.12 487 488 ## Urgent Upgrade Notes 489 490 ### (No, really, you MUST read this before you upgrade) 491 492 - CVE-2020-8559 (Medium): Privilege escalation from compromised node to cluster. See https://github.com/kubernetes/kubernetes/issues/92914 for more details. 493 The API Server will no longer proxy non-101 responses for upgrade requests. This could break proxied backends (such as an extension API server) that respond to upgrade requests with a non-101 response code. ([#92941](https://github.com/kubernetes/kubernetes/pull/92941), [@tallclair](https://github.com/tallclair)) [SIG API Machinery] 494 495 ## Changes by Kind 496 497 ### Bug or Regression 498 499 - CVE-2020-8557 (Medium): Node-local denial of service via container /etc/hosts file. See https://github.com/kubernetes/kubernetes/issues/93032 for more details. ([#92916](https://github.com/kubernetes/kubernetes/pull/92916), [@joelsmith](https://github.com/joelsmith)) [SIG Node] 500 - Extend kube-apiserver /readyz with new "informer-sync" check ensuring that internal informers are synced. ([#92644](https://github.com/kubernetes/kubernetes/pull/92644), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery and Testing] 501 - Fix: GetLabelsForVolume panic issue for azure disk PV ([#92166](https://github.com/kubernetes/kubernetes/pull/92166), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 502 - Fix: use force detach for azure disk ([#91948](https://github.com/kubernetes/kubernetes/pull/91948), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 503 - Fixes a problem with 63-second or 1-second connection delays with some VXLAN-based 504 network plugins which was first widely noticed in 1.16 (though some users saw it 505 earlier than that, possibly only with specific network plugins). If you were previously 506 using ethtool to disable checksum offload on your primary network interface, you should 507 now be able to stop doing that. ([#92035](https://github.com/kubernetes/kubernetes/pull/92035), [@danwinship](https://github.com/danwinship)) [SIG Network and Node] 508 - Kubeadm: add the deprecated flag --port=0 to kube-controller-manager and kube-scheduler manifests to disable insecure serving. Without this flag the components by default serve (e.g. /metrics) insecurely on the default node interface (controlled by --address). Users that wish to override this behavior and enable insecure serving can pass a custom --port=X via kubeadm's "extraArgs" mechanic for these components. ([#92720](https://github.com/kubernetes/kubernetes/pull/92720), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle] 509 - hyperkube: Use debian-hyperkube-base@v1.1.1 image 510 511 Includes iproute2 to fix a regression in hyperkube images 512 when using hyperkube as a kubelet ([#92626](https://github.com/kubernetes/kubernetes/pull/92626), [@justaugustus](https://github.com/justaugustus)) [SIG Cluster Lifecycle, Network and Release] 513 514 ## Dependencies 515 516 ### Added 517 _Nothing has changed._ 518 519 ### Changed 520 _Nothing has changed._ 521 522 ### Removed 523 _Nothing has changed._ 524 525 526 527 # v1.16.12 528 529 530 ## Downloads for v1.16.12 531 532 ### Source Code 533 534 filename | sha512 hash 535 -------- | ----------- 536 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes.tar.gz) | 16aead0c708f1a64ee8b4723551c0e27f3661b06997fe5f32543b9c8f46d89e277d8845248725c64cdfbed9030cdee73e5e6a73acd7775bd5f54f7abab1817a7 537 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-src.tar.gz) | f78c645f6662e0593fa90946df136ad24f76f87ec75ba3131f52eef42c25948b3d49b2139bdb14e73a2bb203489b06e041441c8bf83373bb095bdd6271bcb8e6 538 539 ### Client binaries 540 541 filename | sha512 hash 542 -------- | ----------- 543 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-darwin-386.tar.gz) | d0c8b3d3bfd2a74f2c4335aa7d0f1d51bc5b5249431e88c1e3f83842640b30ec8e4185c4c8cc8be6b51d296e5c62a5cd4363b73c99ff665a9bd0ad609e7b5913 544 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-darwin-amd64.tar.gz) | d4e6b09cc24c04d04bf31510fbbfdc56b63ad2c95a977f22f962f4cd3c1c2ada8e6730ccce1ea3506cd74f9477aa6194221af593c50072b0615ae4c8150a7fa9 545 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-linux-386.tar.gz) | d801441d1e61cf80adea0f556fdc6b6c6b6d3950ae3f5f349d21b45b2ee87635f31b2eff18f491fb95e8f78aebf504809b7695d1ec7dfa5bfa5b47f4c3a624cf 546 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-linux-amd64.tar.gz) | 5f64aebafd1f5c3d2974a7ecf15defb6f82b4a03f4402555bf973a4fde40de3e81b62a92ef4cafee26a40ff6d4c8fc82580c26855ee587835a5f6740d4c89759 547 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-linux-arm.tar.gz) | 49d0c00ac8764c8632d4ffe64b9e0558eb69156bdfbdf56b142082e3ccdc5b4df404600a867de4bd5be40050eefd298fec638b65a7b288d601c0da6c5d97d7b1 548 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-linux-arm64.tar.gz) | 6a8690a22884e9527518e7ba79760b78af426c4b5f6a7b6ceabe7a44658a5459349f6e2342993bcbc82c5b77f3640602a2033ba5390f0c13489bdcbd620834db 549 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-linux-ppc64le.tar.gz) | 8325e3ded3ce32d4c6f778f7247a80900c5bdbb6ce965a83b8a267f60f9508031ab9429b1c622b0026e70ffdd0d1b1384ccb1e0d5b79a059e5c4cfbf8274819c 550 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-linux-s390x.tar.gz) | 160fab60b192c13da14cbb7f35d49d084550401758f7622656b278c09f4b1bb0454ff6507c065f0b446ddbbc92f1a1121c4302203cb032029644009c8c904ac9 551 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-windows-386.tar.gz) | 129d9ec1da5d562693d807f37e73e6a394ebbde9ea421b148eb72ce004e09939530d470d10d118f8894310df4581139a33dd38efcf4c09e7f04f1c58882a08b6 552 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-client-windows-amd64.tar.gz) | 8b8ec10129bfac6490a73898eee8583959f67f43b729a3ef7f55bad05a3b826dbdc6690bfe207260d0bbc8fcb1cc45d995d0046b78dc70c2fa18be984caa8c21 553 554 ### Server binaries 555 556 filename | sha512 hash 557 -------- | ----------- 558 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-server-linux-amd64.tar.gz) | 990903598f44df71f9c31c51be50c1f17449443bd2b2be77344fd7c77d8373eefb8763ea1841d4c831310c94eb7e3de0e1229498ebb70d0e11469d1aa8ca65b0 559 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-server-linux-arm.tar.gz) | 9b3f9a8152d52dd35e1b29da2b3ab88afed8ea93be493c464ba4b11bbd81983686604e178b5e779e95982e42a7ca65ace5c382a34c5809a9903d9efd51c67a0c 560 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-server-linux-arm64.tar.gz) | 9275700fb89441230810d844f63913bb235b4aa4122bb69bf95ba05040175bb176301db2642265caf76dc47ab00919e65020debc973a5e4d339919861d1bc33d 561 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-server-linux-ppc64le.tar.gz) | eb342c35660a3de9991b5e54dbd75c598031a42eb9b0b58f034bbefc401a4c645bc99737025e849acdf34ab06d4b5e16be09141b95d11e3d2cc21c86e27c23dc 562 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-server-linux-s390x.tar.gz) | d52252d386c5387534a62dc26a59c5a8eb175cdedbd79b2d752d9602bb46426f344aedd0f24dda053561f46e172027b4ad8cd67f8472f3cbea8645a3e4138449 563 564 ### Node binaries 565 566 filename | sha512 hash 567 -------- | ----------- 568 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-node-linux-amd64.tar.gz) | 54a5ff3b8e33eb0ca299e758170f8aeb0c931bd7700ac80dde56359ca388e6dc4442dd022bbe1b2a1ecbb8ed92f13624aa41859cbd1cd0a1d2f1a7c86f048d0a 569 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-node-linux-arm.tar.gz) | 494f15950d8ba4e33d71a1dd8c500c78c7e07690b94fe9da87242a8bfe78b3258ff1e522a620449998f36029b4ac013386c800662cda55d4a3933a80c7e9742d 570 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-node-linux-arm64.tar.gz) | 02d87a40a2de13ecb17a30ef80c2b4116cd7690b7aa2e2f685e79d35024f143c131e5943676656f8692a8efb69592f230e4b844690c96e4faff215de4e9d371a 571 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-node-linux-ppc64le.tar.gz) | 1993ad9137602fc706e2d088c17c3c20e80254f450c13f0ccec8c81160da13fbc41c4e7fca22161500494ef62ba3256415fc1f917212c1d5896119a5988ebd67 572 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-node-linux-s390x.tar.gz) | 1cd922e7d15292e3512d501303a08b98133d046467d117323501f0dc477da404b27297e3615c7db26b7e24df75d42ca54569980169be6cbe6f76d3045f4d53db 573 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.12/kubernetes-node-windows-amd64.tar.gz) | 697c10a14cb4f8f021907ec46096c98d3ec69d3d865ad0194a31ae5d3901ba3669f7b82c9b8f08ee7153ba8c730b072ce14dcbe8351db8ca17bbbfd624841a40 574 575 ## Changelog since v1.16.11 576 577 ## Changes by Kind 578 579 ### API Change 580 581 - Fixed: log timestamps now include trailing zeros to maintain a fixed width ([#91207](https://github.com/kubernetes/kubernetes/pull/91207), [@iamchuckss](https://github.com/iamchuckss)) [SIG Apps and Node] 582 583 ### Bug or Regression 584 585 - If firstTimestamp is not set use firstTimestamp or eventTime when printing event ([#91056](https://github.com/kubernetes/kubernetes/pull/91056), [@soltysh](https://github.com/soltysh)) [SIG CLI] 586 - hyperkube: Use debian-hyperkube-base@v1.1.0 image 587 588 A previous release built hyperkube using the debian-hyperkube-base@v1.0.0, 589 which was updated to address a CVE in the CNI plugins. 590 591 A side-effect of using this new image was that the networking packages 592 (namely `iptables`) drifted from the versions used in the kube-proxy images. 593 594 The following issues were filed on kube-proxy failures when using hyperkube: 595 - https://github.com/kubernetes/kubernetes/issues/92275 596 - https://github.com/kubernetes/kubernetes/issues/92272 597 - https://github.com/kubernetes/kubernetes/issues/92250 598 599 To address this, the new debian-hyperkube-base image (v1.1.0) uses the 600 debian-iptables base image (v12.1.0), which includes iptables-wrapper, a 601 script used to determine the correct iptables mode to run in. ([#92495](https://github.com/kubernetes/kubernetes/pull/92495), [@justaugustus](https://github.com/justaugustus)) [SIG Cluster Lifecycle and Release] 602 603 ## Dependencies 604 605 ### Added 606 _Nothing has changed._ 607 608 ### Changed 609 _Nothing has changed._ 610 611 ### Removed 612 _Nothing has changed._ 613 614 615 616 # v1.16.12-rc.1 617 618 619 ## Downloads for v1.16.12-rc.1 620 621 ### Source Code 622 623 filename | sha512 hash 624 -------- | ----------- 625 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes.tar.gz) | 1aea402d8a608320ed420291f560523e2f3cb0c9d748ab9068c65bf41ca7a698bee9d3380951ae60548832196be35150d0de6295ba431d9480cb5b9f431b6916 626 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-src.tar.gz) | 887cef5ebf5958b4d1e924499dbd1d89054bd7bd16326c28dfc3912e880172febbd0037121a1d57b6a39b84e9f0c57a3b4a2b553582947442530b8c8d053db0a 627 628 ### Client binaries 629 630 filename | sha512 hash 631 -------- | ----------- 632 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-darwin-386.tar.gz) | 86323ec3741ac367d937504bafdf14896d5cb086022cad057641d47f39688dc743c3868b4493c976fc3ce0822ce5865889a77234cd2604e65b924c358ac7f89e 633 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-darwin-amd64.tar.gz) | 332c541a3bca1328776a5f27c9f5d6dd162499424bd4fcd1f2ef2f5b68d7ebcecef249cf4f981ecf05d66e3d3c80d85a7e074c5403647428cbca6d3678583db0 634 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-linux-386.tar.gz) | 7a91e23bb34c5ce0c81a625676fa22561cfe09dba292460f28df9f697b0b14d7bbd6688c00aea6c6a213a4dea56b6c7bbf3f136020700e8b01a0449614fada4a 635 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-linux-amd64.tar.gz) | 3da004d95a4e94c6f12bf7323340da4ebe6064582b91ae8d081330a1c578f6de59dfb94aff5ce6f4570c060aa43388116b9cb6193783221340d4a3c957a6fc3c 636 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-linux-arm.tar.gz) | b93385aaf9e9790a6b5277a3f4a2b86f90502ac7309e841d43144df667baa50fc2ebba3de3e261280b807f077905fcaf3b2b7102e12b17d2501e0ad7b639313e 637 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-linux-arm64.tar.gz) | ca94841465a723682fae0c91130fed5fa071a8527e4fa9ce9ed250092d1a82d79e5efdc0f648a89d27e4fcc052db5ae47fad7b8d0ff8403da8564021b6b8ab46 638 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-linux-ppc64le.tar.gz) | b5611661e2bc086ce065067c502ece68004bf95ed4978438b1503caf981698e63a633a03892bf9a4f60fa93b083f5eae5b6798d3b155a8d81f4af556f545fcf6 639 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-linux-s390x.tar.gz) | 06ef2e0b53c288e7cd26576681a51fafe3f864cb196da9a12c2f2ee20be2c571b500f0b1abfa2e0d71c9b6c58bb25fd6944190a5410ad73a15b0b9518c854451 640 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-windows-386.tar.gz) | 48a8d5a4967eeed59167e141bddaf6bce4d09dae13db03742f28037e9d5cb50adb1fbdab91225ae4ea6aff17b50b7ed9349dd027ae1eb6d3457ef2bddd3e37c8 641 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-client-windows-amd64.tar.gz) | 720823ec97ecfe588f428f4c876ea6d8597293d14ecc846e766cde3f1a77e8dd47171084066051b328d86629983ceddee02dc30efabe2884bb041985842b819f 642 643 ### Server binaries 644 645 filename | sha512 hash 646 -------- | ----------- 647 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-server-linux-amd64.tar.gz) | 28fd9bbbd831d129e5f7bc495179b0e6b6aaf21f5e2db8c163cd13406a3916c26d4f94dc9cd844ba7274ee2ec519024019ae403cd3a779c3c8654c5de7e2ec9f 648 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-server-linux-arm.tar.gz) | ad20d7ba5b7c559206aa4253b9ad79e43bf4f054799cb5e241b9675aed04db5102b519e43c58b0c1e7022c77f016a57ba9f72bdd22cd48d8136b1de8e1e23a1b 649 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-server-linux-arm64.tar.gz) | c4d09fbd0576e1048d6ba404a68abb15f57b1af4daf4321b01a8799d9f3696afe6d01b2a956ba203f99e2c2352fef356f7ec4d38bb5fd51d8ea864776d05e81e 650 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-server-linux-ppc64le.tar.gz) | 07085ba7b027be4df44c95b54b8972150bcf9a965fe9a767f8371c71f5746b6214fd8f206a9917e17ac0bf969443f25199eb4a05c8302b43b2fa57b40775b01c 651 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-server-linux-s390x.tar.gz) | b03b616f9e52397bf6c4ea1cfb952b994d3d12c52761d43bd967ebe1857c0d1ce434df69c51ac9695495f1bd074ad5e739407a1c5a1adc4650b019231a344cad 652 653 ### Node binaries 654 655 filename | sha512 hash 656 -------- | ----------- 657 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-node-linux-amd64.tar.gz) | 8b15546050327feed884c2f05b0d2b34d95d475688e2c1a809f964f47b6f16a59e31d0caafc8b09f46301d0a6fd9c411be9469bbbea626d78dbe17f9988e03c8 658 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-node-linux-arm.tar.gz) | 38555ae95278a585d1c104c9198d6b9504ceff5c750e1b8fe2d00d3c2846ffb8ff4f3d3cd31ad4dc328c1d026537007de32958c85113a48bede6be727aea047d 659 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-node-linux-arm64.tar.gz) | 047b53bc59737b142cc9a3098ae284ed38c8f80f3d9cd747c25eef3f5871e6cd5d1e5443f521dea4b7ffebd7495110cbe47b293b8394b9636ac7c2d9ec7be1df 660 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-node-linux-ppc64le.tar.gz) | 05b138594fcbbb0ffd531b78e0b09fa2300b8fbf013c88f9d0f083b392f7a3642dfb8ebadae79e3a5f0a85ddac35f0e88c11a2e62e2bd335b8ca85a64a7fed96 661 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-node-linux-s390x.tar.gz) | f29940c3063d9e738dbe77b6813940efb34b333197464557664a232ee6e5adc075a9a206cab00ff2f478230da5f20e6055e8802b336b0ecd3764d3641b48d7e3 662 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.12-rc.1/kubernetes-node-windows-amd64.tar.gz) | 381a3065d15737b6e1a6fbd59ad66a41cc55d84a550c3037137ad64dc535a371876fb85fddda97742f2402fe9ec6e6ab67013aa49ac7ada224dba3b565b89214 663 664 ## Changelog since v1.16.11 665 666 ## Changes by Kind 667 668 ### API Change 669 670 - Fixed: log timestamps now include trailing zeros to maintain a fixed width ([#91207](https://github.com/kubernetes/kubernetes/pull/91207), [@iamchuckss](https://github.com/iamchuckss)) [SIG Apps and Node] 671 672 ### Bug or Regression 673 674 - If firstTimestamp is not set use firstTimestamp or eventTime when printing event ([#91056](https://github.com/kubernetes/kubernetes/pull/91056), [@soltysh](https://github.com/soltysh)) [SIG CLI] 675 - hyperkube: Use debian-hyperkube-base@v1.1.0 image 676 677 A previous release built hyperkube using the debian-hyperkube-base@v1.0.0, 678 which was updated to address a CVE in the CNI plugins. 679 680 A side-effect of using this new image was that the networking packages 681 (namely `iptables`) drifted from the versions used in the kube-proxy images. 682 683 The following issues were filed on kube-proxy failures when using hyperkube: 684 - https://github.com/kubernetes/kubernetes/issues/92275 685 - https://github.com/kubernetes/kubernetes/issues/92272 686 - https://github.com/kubernetes/kubernetes/issues/92250 687 688 To address this, the new debian-hyperkube-base image (v1.1.0) uses the 689 debian-iptables base image (v12.1.0), which includes iptables-wrapper, a 690 script used to determine the correct iptables mode to run in. ([#92495](https://github.com/kubernetes/kubernetes/pull/92495), [@justaugustus](https://github.com/justaugustus)) [SIG Cluster Lifecycle and Release] 691 692 ## Dependencies 693 694 ### Added 695 _Nothing has changed._ 696 697 ### Changed 698 _Nothing has changed._ 699 700 ### Removed 701 _Nothing has changed._ 702 703 704 705 # v1.16.11 706 707 708 ## Downloads for v1.16.11 709 710 ### Source Code 711 712 filename | sha512 hash 713 -------- | ----------- 714 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes.tar.gz) | 3830d26b53cc18be80a40249f9db01bde72d3f0f222252cfccd46416ec1725527dcc04aa85f8af4b2847c041807b08b9e4b78a6748b589592399a15160e2467e 715 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-src.tar.gz) | 86c376157342758e346bfd028ff7d8dfd0a40afe96c4d77835c2bd71ef999643c9e1961a9549caf3595bd158dc025e858183544cf4ccac59d5951fa440f93802 716 717 ### Client binaries 718 719 filename | sha512 hash 720 -------- | ----------- 721 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-darwin-386.tar.gz) | f8147e6dfc8348c59848fdabbb8ce89e9b7f7c4999de26c82c6caa2cafda24cce8b89dcaa53caf3c39fb319104ec5203482127daec68aa2cd629a6c29a50e747 722 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-darwin-amd64.tar.gz) | 35c45d7ac5ba281daa4b504f1256b9b67a1cb3476b394ce2b19f5363fb9da4ecac5f923fbe7d7ade3d636bde3646965424408b4a7b6f1ee0a2892383419f8d97 723 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-linux-386.tar.gz) | 2d185a2f3daf3ea0ff7e0b3eec98c3159d238db5be4dc8423e8ecfbff9d1b6ca17dc7ee3cfff8a52c8cccecc27ae5a2c393fa84c1c2237336ee898c409e0fdd8 724 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-linux-amd64.tar.gz) | 5181f29e832a46bcd454c812cfbabb7a0786434796d893086c29013e59288e56a611892fe4f85d9ba56a70699bf26e50ab61acbe6eb99b06eddd2a6fe4841824 725 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-linux-arm.tar.gz) | 7c87111b4bdf94b1305551bbc9e4aab1edf660867a1346f133ef079354c8c7b8af71fd1ebf46eb8544a887cf0e1dba3fdf0dcdb40fbd9fecff58b098c2491b2e 726 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-linux-arm64.tar.gz) | 2241c28ab4d08338cadc43862d0daa235158237cf5e3caa0d7d755ac0cb44039c0200e14abd2e41454d5880b8ff11c811dff0c05e88405b04192f1f342914dad 727 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-linux-ppc64le.tar.gz) | a052f627964db4c68268a37afda32af637f74876f254633de16f0e56fe1c3fbba13e67a8a0c4f2a475c5cb69038c6fff545a50e8a9658eb20037020a96cf08b8 728 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-linux-s390x.tar.gz) | cd43ef6f58451e5b4a5c02b27beefd69b6b08bf4846b920c83e9886cb17dce72d358b7d138515454d9c802f327edb3e9b17e8a4319a19b806fdca0f2a857f9b0 729 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-windows-386.tar.gz) | 94108a54806d28edf6e0f75124bfe192be415e6245e03963c93cc20b9b8eeec48d66e4b7c4e99fca5b6e23628f2925f6c163d910614260c4c1bd32274abb9a3e 730 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-client-windows-amd64.tar.gz) | f5139a7ceb8f3f9f07bd5ab2f6762b166152e886a78149264789b7f6cad15773e76d4acb42f46c2c6af7c4ec4e8c30f32ce089c92d28cab74e6f6f978f839d4b 731 732 ### Server binaries 733 734 filename | sha512 hash 735 -------- | ----------- 736 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-server-linux-amd64.tar.gz) | 548e9d9c09458ecd3a5ec83560ecc53fce589edf85b27776d0f501547492a46405be33997e740c1d69dddda3038d8aed7886a26364c9925c50e54aabe4a035fa 737 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-server-linux-arm.tar.gz) | 3858a3a1f5e5edaa48e4ccae37936426222d440802c195923cb29bc02bcf7e2fb18928e6084bb93c20deb59898e7d9ecf43fc8702049caea83233d79bd438456 738 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-server-linux-arm64.tar.gz) | 7ab1a2018eb190385744799af48362ec9fbde5cff42433b71aa73700033103e3c67a168d981b636da81f712fa01eda73a2010c3de5768e2333b1c0cfce3ccc5f 739 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-server-linux-ppc64le.tar.gz) | e6463da01ac934cc826ae781281737cfe6f42b4355d68e0b90a589af0e14ece1ad59523799b7470ae9d49a735137a6d371933ccc04de6a7c99731ca6822cfb0d 740 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-server-linux-s390x.tar.gz) | 8be2923df4fa559b647550aba78d3e0a82b1c3d22d7e7043424ce265c9bff593cb7c78d3d3e92d8a973fc4df5a4f118edba3c50ae32ee54fa56e220d1bf9a10b 741 742 ### Node binaries 743 744 filename | sha512 hash 745 -------- | ----------- 746 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-node-linux-amd64.tar.gz) | ba0f723f05201a0ce7cb8655484f530e22959c21ef7da32eaab95741b97fd9487622bdc229c7b02c14978301dff020939cf88e31f1032110c6722b76c6a7c0a9 747 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-node-linux-arm.tar.gz) | e81998954ca0349169c76385f57dc0bf480f803e191f79c15cbc2eed4ed5f59ca3c991fe0316b590624518b80be2d790c2bb6ba0421854cc84226d74d725a5ed 748 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-node-linux-arm64.tar.gz) | afbc62f35d7f36a424edbb728b4c1030a594e81666137bff566f7de824b9c4479c8e3a18bf827257d1b14f6aa33efdf27f1259eef014c37742e1a29a84891ee3 749 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-node-linux-ppc64le.tar.gz) | ae544dc78ec673aaabae89e0542579eca19344edac2b830d280cc763c583a84c0978725ac162c875e1d237db1e763d27d2d9ebfa3b272959ccc57676ac28bf49 750 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-node-linux-s390x.tar.gz) | 727d3aa9fe8a4a3e7cfa2b151afe5e43a94579b55c7c5e603659a686349dc4141a8b8df9d92bb4fe9078f2b9fc877fc02afc6dc30826a5ac0d3d9f4a74e641f6 751 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.11/kubernetes-node-windows-amd64.tar.gz) | a7dfa7f10868b061dacde037d1fb8763387724310271cb4cf8d3a5a270a0dc1c52beda81abbdf934217634c8c253d2d48d43ece4538863016a68db6301730bfb 752 753 ## Changelog since v1.16.10 754 755 ## Changes by Kind 756 757 ### API Change 758 759 - Resolve regression in metadata.managedFields handling in update/patch requests submitted by older API clients ([#91748](https://github.com/kubernetes/kubernetes/pull/91748), [@apelisse](https://github.com/apelisse)) [SIG API Machinery and Testing] 760 761 ### Feature 762 763 - Extend AWS azToRegion method to support Local Zones ([#90874](https://github.com/kubernetes/kubernetes/pull/90874), [@Jeffwan](https://github.com/Jeffwan)) [SIG Cloud Provider] 764 765 ### Bug or Regression 766 767 - Azure: set dest prefix and port for IPv6 inbound security rule ([#91831](https://github.com/kubernetes/kubernetes/pull/91831), [@aramase](https://github.com/aramase)) [SIG Cloud Provider] 768 - Fix internal loadbalancer configuration failure when subnet name too long ([#86276](https://github.com/kubernetes/kubernetes/pull/86276), [@yangl900](https://github.com/yangl900)) [SIG Cloud Provider] 769 - Fix public IP not shown issues after assigning public IP to Azure VMs ([#90886](https://github.com/kubernetes/kubernetes/pull/90886), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 770 - Fixed a regression preventing garbage collection of RBAC role and binding objects ([#90534](https://github.com/kubernetes/kubernetes/pull/90534), [@apelisse](https://github.com/apelisse)) [SIG Auth] 771 - Fixed cleanup of raw block devices after kubelet restart. ([#83451](https://github.com/kubernetes/kubernetes/pull/83451), [@jsafrane](https://github.com/jsafrane)) [SIG Node, Storage and Testing] 772 - Kubelet podresources API now provides the information about active pods only. ([#79409](https://github.com/kubernetes/kubernetes/pull/79409), [@takmatsu](https://github.com/takmatsu)) [SIG Node] 773 - Pod Finalizers and Conditions updates are skipped for re-scheduling attempts ([#91953](https://github.com/kubernetes/kubernetes/pull/91953), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] 774 - Resolve regression in metadata.managedFields handling in create/update/patch requests not using server-side apply ([#91690](https://github.com/kubernetes/kubernetes/pull/91690), [@apelisse](https://github.com/apelisse)) [SIG API Machinery and Testing] 775 - Resolves an issue using `kubectl certificate approve/deny` against a server serving the v1 CSR API ([#91691](https://github.com/kubernetes/kubernetes/pull/91691), [@liggitt](https://github.com/liggitt)) [SIG Auth and CLI] 776 777 ### Other (Cleanup or Flake) 778 779 - Update CNI to v0.8.6 780 - build: Use debian-hyperkube-base@v1.0.0 image ([#91388](https://github.com/kubernetes/kubernetes/pull/91388), [@justaugustus](https://github.com/justaugustus)) [SIG Cluster Lifecycle, Network, Release and Testing] 781 782 ## Dependencies 783 784 ### Added 785 _Nothing has changed._ 786 787 ### Changed 788 _Nothing has changed._ 789 790 ### Removed 791 _Nothing has changed._ 792 793 794 795 # v1.16.10 796 797 798 ## Downloads for v1.16.10 799 800 ### Source Code 801 802 filename | sha512 hash 803 -------- | ----------- 804 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes.tar.gz) | 5925579630fa2c3f78c1e17b4c2aca3e23cad5db7721b34b9e8d8eb8a443d4ea4cb87369d9d254dfd35603a1f12825a1167ccd2b71e1e1f3eb7676ad4a143276 805 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-src.tar.gz) | 5d265121125e940ea43a54df7ef6ed072598d5d4dcfcbf7173ad252d016869db938c50865aaff31ef7ebd0399fa0de87b4248285b3c7f1abdf813cda0df585b2 806 807 ### Client binaries 808 809 filename | sha512 hash 810 -------- | ----------- 811 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-darwin-386.tar.gz) | da77e3fc2bdcc5c6c6dd8c92c2f8fc23ea2974cb36e08316830631132005c03513296179b1ba5e77d9dc3346ce273207f9ae4ae822b13b1a361f6543a4df4bcc 812 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-darwin-amd64.tar.gz) | f56f42a525e723fdcb203d7671fc8586c45a9987e8f4ffbe9467beefe0af12f8158bba1257392f21cca3cf75c9f8db803d644135da5e2fb2cc74ced858c9f294 813 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-linux-386.tar.gz) | 10241604f0fec126f696f33963ec607abf378b44524f3dcc3ab26cfcad894d1ab1f3fa06af2ee6619ac3aa43eef6edb46bb7cdb3e5c9e813009a5373b99e1598 814 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-linux-amd64.tar.gz) | 455d8bd8881aa996dd35969b2448fa54ab15650ce2d58c19bf89ff8e11f287beb136ba64ca62d64c958a1227dfee1e94408941258f0b9421ef06bb48a09bbcd2 815 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-linux-arm.tar.gz) | a30b1840f7ca2f13468a142f69b1806cf9950d67f5c42cb30972c23704c4c66d47c16245096ddb3245ab9d7a002ae37a944822ef4d8d7d21a253f8cf59d861c8 816 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-linux-arm64.tar.gz) | 1247c48d1abe3d04d8ecc08c34bc627961cc2a830893a039366e36da66ce96431d2d9b8ebb0e4893260195ba702f8a5e9935ac5a043ea0a9b0e34b64682f394b 817 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-linux-ppc64le.tar.gz) | c793de5dbe35708ba7992cae395e3c70e48e5ecb57bba73d87496dbccde216b777f60ff41ff10d565e06f48bf18b72f97df12446722860a72a55c46dcdf8bd7e 818 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-linux-s390x.tar.gz) | a71420474b115e5114628b09744a1929163931550d5fbfffbc6c12c7531c6e9769ab5125148a72d151fe6392c8c96d6a4736348a67fa02e66ac8afac6defe13b 819 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-windows-386.tar.gz) | 42a44700becf4556473498c95561aced6704a49800bf1b96a776607903eca97f77aa208e29d99c963a7e00a8af72540c45572f1fad59d74f3736b180e60f7411 820 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-client-windows-amd64.tar.gz) | 0fffcf5012ae9495396c64a8b3bea3389456ba5844d44b5b8ae619752011101864eb7ed6ea6fe2fedeb8a07e36a097cd84b79a62b9004e457105f0c0895f7be8 821 822 ### Server binaries 823 824 filename | sha512 hash 825 -------- | ----------- 826 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-server-linux-amd64.tar.gz) | bb2c68409878c1037f8d2924b010f6e13cb28969046faf75a5e9e16d90e8167223f9ecab0866ba1c3998360d1da87863d3626dada226b153733f39e6d53e710c 827 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-server-linux-arm.tar.gz) | 42dddd7699b77c12690b525dd9c0643e529589a84628c3fa6b7f8ffad064dc1fb93cd3e37aa606d113f4c18833ad6e404af86f0dba5b92238c2410f9bdeb16fa 828 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-server-linux-arm64.tar.gz) | 0c0e9b484576559eb2d894aef9b705472d31b459123ca4dec61fa0fd9a6d5ddb844003dfbbaa596e3d60fbabfdbd57424053a89c0d9969d181e5e9b820c49d86 829 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-server-linux-ppc64le.tar.gz) | 9a6c12f1c6a442d59f8eb2affa3c980e35c0e85a70ec97d565c660c4bf041257d98f8588280a1e4b28f9a2589c5f8f5ef105030a29271c4bd0faf67b79eb37b6 830 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-server-linux-s390x.tar.gz) | f4a272179806dc174177c4ae860a8f8dd67743ed1eedab2720761a008e8bf15646c39e0a8a176e980a2e43d2c8068f78f19c35295ceb097e40eea904d5e6c253 831 832 ### Node binaries 833 834 filename | sha512 hash 835 -------- | ----------- 836 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-node-linux-amd64.tar.gz) | bcd8abebedbb20f93259ccfddefe4f7f5583d2572a8bdb011715b737c88927a89f90923856e43ed6b6b9d6fdd7ce6f348de5e632f63abe16ee66828a33842844 837 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-node-linux-arm.tar.gz) | 9089881d92640de936c07b25ed398e338d95a0b684891377835341e47f868fa0f3337ac8e93307937d2ccdb694da72fc6ca275f1bb2803ae55fe06c7a1d47cd2 838 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-node-linux-arm64.tar.gz) | b5844cfcbdc13888c3150348252cdf38c39a19fc0e69b1bff42178e10b608e8500e7e527a285c20be0e9ca1ea785c7ff9a198c9e151e7e9ebed694fc7780214c 839 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-node-linux-ppc64le.tar.gz) | bda2e59f704cb314397302a3365f8a2dcbd7663277ede11fe5f69c346ab5ef26103542849751a3c8cd2a72c9819cbc4900b52a0f941d47fb7d774caaa94fda24 840 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-node-linux-s390x.tar.gz) | 0574ccf4677f8333b6a64c91fcfc4e3d037eec05f3983df647d8d5505ef57a3c6d41e93fbb35ad1f228b3673ee0d6084610054e397714ba0bd86c949455122d9 841 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.10/kubernetes-node-windows-amd64.tar.gz) | e5a5bfb79f1e45f13da3dfaa13c44264e099d08c535d6a9c63819685be186f8a84a518468720cd56dc1434aec10d40aa4c929d62fdd7f43b749e05c7572fb2f9 842 843 ## Changelog since v1.16.9 844 845 ## Changes by Kind 846 847 ### API Change 848 - Fix bug where sending a status update completely wipes managedFields for some types. ([#90033](https://github.com/kubernetes/kubernetes/pull/90033), [@apelisse](https://github.com/apelisse)) [SIG API Machinery and Testing] 849 850 ### Bug or Regression 851 - Base-images: Update to kube-cross:v1.13.9-5 ([#90966](https://github.com/kubernetes/kubernetes/pull/90966), [@justaugustus](https://github.com/justaugustus)) [SIG Release] 852 - Fix HPA when using init containers and CRI-O ([#90900](https://github.com/kubernetes/kubernetes/pull/90900), [@joelsmith](https://github.com/joelsmith)) [SIG Node] 853 - Fix: Init containers are now considered for the calculation of resource requests when scheduling ([#90455](https://github.com/kubernetes/kubernetes/pull/90455), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] 854 - Fix: azure disk dangling attach issue which would cause API throttling ([#90749](https://github.com/kubernetes/kubernetes/pull/90749), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 855 - Fix: get attach disk error due to missing item in max count table ([#89768](https://github.com/kubernetes/kubernetes/pull/89768), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 856 - Fix: support removal of nodes backed by deleted non VMSS instances on Azure ([#91184](https://github.com/kubernetes/kubernetes/pull/91184), [@bpineau](https://github.com/bpineau)) [SIG Cloud Provider] 857 - Fixes a bug defining a default value for a replicas field in a custom resource definition that has the scale subresource enabled ([#90022](https://github.com/kubernetes/kubernetes/pull/90022), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, CLI, Cloud Provider and Cluster Lifecycle] 858 - Pods that are considered for preemption and haven't started don't produce an error log. ([#90241](https://github.com/kubernetes/kubernetes/pull/90241), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] 859 - Provides a fix to allow a cluster in a private Azure cloud to authenticate to ACR in the same cloud. ([#90425](https://github.com/kubernetes/kubernetes/pull/90425), [@DavidParks8](https://github.com/DavidParks8)) [SIG Cloud Provider] 860 - Reduced frequency of DescribeVolumes calls of AWS API when attaching/detaching a volume. 861 Fixed "requested device X but found Y" attach error on AWS. ([#89894](https://github.com/kubernetes/kubernetes/pull/89894), [@johanneswuerbach](https://github.com/johanneswuerbach)) [SIG Cloud Provider] 862 - Scheduling failures due to no nodes available are now reported as unschedulable under ```schedule_attempts_total``` metric. ([#90989](https://github.com/kubernetes/kubernetes/pull/90989), [@ahg-g](https://github.com/ahg-g)) [SIG Scheduling] 863 864 ### Other (Cleanup or Flake) 865 - base-images: Use debian-base:v2.1.0 (update to Debian Buster, includes CVE fixes) 866 - base-images: Use debian-iptables:v12.1.0 (update to Debian Buster, includes CVE fixes and iptables-nft/iptables-legacy wrapper) ([#90940](https://github.com/kubernetes/kubernetes/pull/90940), [@justaugustus](https://github.com/justaugustus)) [SIG API Machinery, Cluster Lifecycle and Release] 867 868 ## Dependencies 869 870 ### Added 871 _Nothing has changed._ 872 873 ### Changed 874 - k8s.io/kube-openapi: 743ec37 → 594e756 875 876 ### Removed 877 _Nothing has changed._ 878 879 880 881 # v1.16.9 882 883 [Documentation](https://docs.k8s.io) 884 885 ## Downloads for v1.16.9 886 887 filename | sha512 hash 888 -------- | ----------- 889 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes.tar.gz) | `524cb2d5a748e4e0b78aec357b479b8b810fa3bf875b85682d30fe5e4e3d5c6b0a90d0a0ab4dd9866e73a9d78fdbc67de9903bf069e4ee5e303997355e891a54` 890 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-src.tar.gz) | `a5b1d1cbf871d11838ff5be961f22dfaa564f8f3e938571fc0ff87639cfb16193ecdb34ccba922f69f80769fb3f2cd78655fad3a3f4d3f5ebfe9834afbfc5293` 891 892 ### Client Binaries 893 894 filename | sha512 hash 895 -------- | ----------- 896 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-darwin-386.tar.gz) | `a876d5eba6955dcf868c6cb105c39a80dea4eea8de88aef464c8619bbd2a002d88598d50697d8a7d1490d21b3184b6107132bcb821d49da1e469a1ad19b549e4` 897 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-darwin-amd64.tar.gz) | `fad0cc9f88371c909490aa3430d060a8841cceebbb981b404447f933aeb8d4f35107a38f402705bbc7e7741a4a14c34bb6b8078b536865b3708329348e0a79c1` 898 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-linux-386.tar.gz) | `2dcde1c42b85875bd90b04a9301a73144c0b0f768f585c6bb501f327e0652af62fad8d7a029d6753d6bcb9108b67a34725656906d8c065813fd0d9c9b84d5240` 899 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-linux-amd64.tar.gz) | `c2e69794ad86799cf42aefa26f84d1353984c27eb6a5f360388c6bbfe9389ca5e1cf41a4dabf04a9d2b36557a7054a4623bf2c845508b483a982e77827f06629` 900 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-linux-arm.tar.gz) | `876aea3c3ab73765efa6eb657219b36501254a66d36b98274d730035b11cba48aaa9904181afb2df403f176c19d944564455d7af7d918a81dcc83cad07cb70d2` 901 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-linux-arm64.tar.gz) | `d2892e6d81af5ed77186f620a67bf2fc4358d33af8fd818eb6952114df924d834d40efd12ba91e3cce07c664f3f7e902cf0413d86335b6b40c6f53b170ccf66f` 902 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-linux-ppc64le.tar.gz) | `dfbf993cad3f8b65c0addaa5f7d5e74490ef4e77abfeafc1111fea5a86a649bb5ad0300871c64270c5d6e6a779b000442fc7b56447565be18266c8088f65e1f4` 903 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-linux-s390x.tar.gz) | `aa5a41350682300f1cb61230dd26f54135aa74eb537754ea8bdf7aff76a3d534c08730170ba80033c0c4d3628d22122bbd9c7b7d6768ac674cfa980cf0c04395` 904 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-windows-386.tar.gz) | `6e81194ac21e37fd708371ef2df5a5c1276716dd7c9ac71023b7a53e8ba61e7e743fc93eafe0fe4d3be0ec521904dd0db98ea223dbfc3bc89536d9015701eaab` 905 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-client-windows-amd64.tar.gz) | `79f9cdd12cb97b452fc0c66bb812eb0b78278714432c8b801ff3aef96da877fc5b6ed265bd58ae954a56fea9ec4a664fbaff50dacc76d632daa9671b05f8ea1b` 906 907 ### Server Binaries 908 909 filename | sha512 hash 910 -------- | ----------- 911 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-server-linux-amd64.tar.gz) | `79d6e3f984134ede83e7ae1f9b072969a4e43d834815f772eba852c77eccc765b24a68f79514c0074cc3240c796d0be9b166957b7bc539394ded5f5c75cc9ea3` 912 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-server-linux-arm.tar.gz) | `7737906093c273324e0598ac60742ad748023d69c1d7bd151d0643d973a99c163fe309210e3f395f600ed77a4e1f39dc6c3db945728df10a0ebce706e8ee902e` 913 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-server-linux-arm64.tar.gz) | `ff0138d2c91098b380c057db1562d90628806a0508b71ee702bc0608fbd460518cf47064338b569c3c2ff750b69ce6c1c9e6a6649865bed6d834325350e5bc83` 914 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-server-linux-ppc64le.tar.gz) | `b5d8f9271c9ab56a6417a54ff8a51772a465b2ef88713cea5349270c0b56c1e05e09033e69ebb4de3d73975f2ae100d56072e8af7f161f1c7f8e1d7348f25423` 915 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-server-linux-s390x.tar.gz) | `3183d105eb334a2c82d5344b3b77d504daa5ec3310c0268e297014fe624f655febe7a7b515d28a8ffd19ceb083e5306dd2774e9b1623f2aec75983c11f913e49` 916 917 ### Node Binaries 918 919 filename | sha512 hash 920 -------- | ----------- 921 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-node-linux-amd64.tar.gz) | `84a3c9e36835f8efa807569556a228c4a99582e1e559733152a6c02dd8a7b1bcdd3d5c722d34dcffbca898813b7d69cb9e1a338cbeb67a8142b91e091ee7b5ef` 922 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-node-linux-arm.tar.gz) | `4b68a56141351dc2bf5f7368b4cc7caa4f6320bfc98d9cf45d246ef7083bec2f90273b2fee700fdcb10e5123463bf2352af515be2a0b28b410a6a4f5358bcaa7` 923 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-node-linux-arm64.tar.gz) | `798928c0904e1b02c5fb2963e8b9402f0847866a326765d070a27a9ea45c333f004f7b39a31dee8ac7d7bb2b40717419d4b0601346c78ac5ff39905211874c08` 924 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-node-linux-ppc64le.tar.gz) | `b417782b3d18a655ca4c52ce64eb8584e5e71ccefe6989c17d02e553f2f8e0be338ec5a0791e19439e48fe50562fda641828b73deb4e48239ae5a94dfdfb8d56` 925 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-node-linux-s390x.tar.gz) | `3cff6b2fc22a29204c4d4dffd4e5f54e9f551f3adeaf512463523c1dfd1771316fb5da83b02a0dd1c124f4ef402d674b30bcc1b8c634ce61450f06c4f31f10db` 926 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.9/kubernetes-node-windows-amd64.tar.gz) | `0e178894edde44dd43fe9aac94b2ff51c7ba239c0a3827c8dddac4970c361e3d72e44848af731c8b443c25a9ca3ca96b7aa24f42505fe27ccc401fc8e21050ec` 927 928 ## Changelog since v1.16.8 929 930 ## Changes by Kind 931 932 ### Feature 933 934 - deps: Update to Golang 1.13.9 935 - build: Remove kube-cross image building ([#89400](https://github.com/kubernetes/kubernetes/pull/89400), [@justaugustus](https://github.com/justaugustus)) [SIG Release and Testing] 936 937 ### Bug or Regression 938 939 - Client-go: resolves an issue with informers falling back to full list requests when timeouts are encountered, rather than re-establishing a watch. ([#89977](https://github.com/kubernetes/kubernetes/pull/89977), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Testing] 940 - Ensure Azure availability zone is always in lower cases. ([#89722](https://github.com/kubernetes/kubernetes/pull/89722), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 941 - Fix invalid VMSS updates due to incorrect cache ([#89002](https://github.com/kubernetes/kubernetes/pull/89002), [@ArchangelSDY](https://github.com/ArchangelSDY)) [SIG Cloud Provider] 942 - Fix panic in kubelet when running IPv4/IPv6 dual-stack mode with a CNI plugin ([#82508](https://github.com/kubernetes/kubernetes/pull/82508), [@aanm](https://github.com/aanm)) [SIG Network and Node] 943 - Fix the VMSS name and resource group name when updating Azure VMSS for LoadBalancer backendPools ([#89337](https://github.com/kubernetes/kubernetes/pull/89337), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 944 - Fix: check disk status before delete azure disk ([#88360](https://github.com/kubernetes/kubernetes/pull/88360), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 945 - Fixed a data race in kubelet image manager that can cause static pod workers to silently stop working. ([#88915](https://github.com/kubernetes/kubernetes/pull/88915), [@roycaihw](https://github.com/roycaihw)) [SIG Node] 946 - Fixed an issue that could cause the kubelet to incorrectly run concurrent pod reconciliation loops and crash. ([#89055](https://github.com/kubernetes/kubernetes/pull/89055), [@tedyu](https://github.com/tedyu)) [SIG Node] 947 - Fixes conversion error for HorizontalPodAutoscaler objects with invalid annotations ([#89968](https://github.com/kubernetes/kubernetes/pull/89968), [@liggitt](https://github.com/liggitt)) [SIG Autoscaling] 948 - Fixes conversion error in multi-version custom resources that could cause metadata.generation to increment on no-op patches or updates of a custom resource. ([#88995](https://github.com/kubernetes/kubernetes/pull/88995), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 949 - For GCE cluster provider, fix bug of not being able to create internal type load balancer for clusters with more than 1000 nodes in a single zone. ([#89902](https://github.com/kubernetes/kubernetes/pull/89902), [@wojtek-t](https://github.com/wojtek-t)) [SIG Cloud Provider, Network and Scalability] 950 - For volumes that allow attaches across multiple nodes, attach and detach operations across different nodes are now executed in parallel. ([#89240](https://github.com/kubernetes/kubernetes/pull/89240), [@verult](https://github.com/verult)) [SIG Apps, Node and Storage] 951 - Kubelet metrics gathered through metrics-server or prometheus should no longer timeout for Windows nodes running more than 3 pods. ([#87730](https://github.com/kubernetes/kubernetes/pull/87730), [@marosset](https://github.com/marosset)) [SIG Node, Testing and Windows] 952 - Restores priority of static control plane pods in the cluster/gce/manifests control-plane manifests ([#89970](https://github.com/kubernetes/kubernetes/pull/89970), [@liggitt](https://github.com/liggitt)) [SIG Cluster Lifecycle and Node] 953 954 ### Other (Cleanup or Flake) 955 956 - Reduce event spam during a volume operation error. ([#89794](https://github.com/kubernetes/kubernetes/pull/89794), [@msau42](https://github.com/msau42)) [SIG Storage] 957 958 959 # v1.16.8 960 961 [Documentation](https://docs.k8s.io) 962 963 ## Downloads for v1.16.8 964 965 filename | sha512 hash 966 -------- | ----------- 967 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes.tar.gz) | `405aab246b6b49fa6654f982a11af0118ca86d33e84975f26c26e2f09a0781f7578d4ac90f2c889f12c600a93cdc7c57e91e29df6f67b3822573faba889f8fde` 968 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-src.tar.gz) | `553ad557dec7664ac910a2011c563c2f2960add05d88135a4832a601bea5afdf4145d30c64c27f2d95f3f7b33d9a401f1cd48a5fcba67ed5b45adbb493339f1f` 969 970 ### Client Binaries 971 972 filename | sha512 hash 973 -------- | ----------- 974 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-darwin-386.tar.gz) | `b16757163f07c44dc75464b9d08bd175c2e6c2d079455c83ffeefcbb65fe17ea638d6e5ce5106983f747414f63c5c8884db8ee28d284d96a1cfdada664928787` 975 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-darwin-amd64.tar.gz) | `9c8b1876385bf7c5f4cccf1f463b42ed588831e4168623e6076ae3709195398d53ab5b73c9a123d12caa26275fd889f3517b30f024038a96d98061568ee3484a` 976 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-linux-386.tar.gz) | `64760d8861d12f79b78b09d8da02bddb6e3f4b069c4ba5e5e99cf057dc94741db06c91bfc5cc90e0194f4adaed52336c1676b5f35fa48e7d98c7553bb3443a29` 977 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-linux-amd64.tar.gz) | `cbe62c70acd576ad3a84c34df50bb8c9f6245662b185f64064f5323da0b9d712c426f255458f6761d39fa004b22b86e773906d54f7e7f0489a02672fc0fab6f6` 978 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-linux-arm.tar.gz) | `8d84e5c19299d8d7ed4b064a3cac2f9168b6bd3d867f7a1d18f20ac122a071d3513e150efa1d1763e99b2d2c4d1c27b06e8daf80ad4daa60a98ea99a9f15884f` 979 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-linux-arm64.tar.gz) | `e2d1c29733ee713788e6b863f4cbbb9064db788fa2a96d4307bb59d9d96bd5caf6932a04fc6e291e1031079725452e40774647ed9c23250faa1c6edc151036fe` 980 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-linux-ppc64le.tar.gz) | `54d749f7582614bc1b44ae41e001d8ec9073f764f15c4b243cbbb1d384ee4f5dc910ebb1cf4d0e46e6d5491f3834e1a793966a0b53786a76a2477948bd9baac1` 981 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-linux-s390x.tar.gz) | `e3d5036e77f919a1f0270d5c8342e615dd411ed19cc766844a55872cd903892086ae50f247aea469b671cc45ebc78818023e0bb42db4d3e5f2efa9298f2e080b` 982 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-windows-386.tar.gz) | `f845e73d36e222c60f04b614a800af46d1ed7b6775300cde7bd2188067a8e01896cb452d171b187c399c568cb6a87a36837488b50556fb1f053a81efd4e16a8d` 983 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-client-windows-amd64.tar.gz) | `66ab87a9156c8ea290ccded3511e81048e1f99a8db713ad2f15c2af68ac55c74ec4aa718e7835cbf12e97be43ca9bf3e770cbae321d5e3270af42b0d778f6f47` 984 985 ### Server Binaries 986 987 filename | sha512 hash 988 -------- | ----------- 989 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-server-linux-amd64.tar.gz) | `335ade6adf7704b232e0bea7c6556d46d235d0c963a04c0dda8b783c91a06a5b7656226bc977234dfbd596d43ee6183291f969110fb74dc70df1eacd4bb633a7` 990 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-server-linux-arm.tar.gz) | `7902321cad51c10104d9346939df3f0377c71da40fccc9d459b3cdd83ce96fcdcc69dfe4a5cbad05cfc2eaa0655e4fb197fff4a04122808f06f1b62f7f290828` 991 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-server-linux-arm64.tar.gz) | `82c57aa319237a26c5e97cf3d8394f8f71475f5c307c5cb244cd70aca5e5c87a3a1397b9d1f1ebf864a5f32cd2f0427ce38b2ae41003c08a7a145b29fa470545` 992 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-server-linux-ppc64le.tar.gz) | `971836cda047974048f55db5478e5c250ded6b55077bb73cf05f53e7a9be0fa88ab8003064c6bea1c002818ef18f72fe72cb3e25ae89b3fe062086ee9f8d06ae` 993 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-server-linux-s390x.tar.gz) | `6595619e59cf26725c4e5570226931e544c895e1e75dc2d9779ae8f222c68dfbaebe080f582634cc3398cd158b4cb489ff45ac412756a20755d57f9471f2dd76` 994 995 ### Node Binaries 996 997 filename | sha512 hash 998 -------- | ----------- 999 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-node-linux-amd64.tar.gz) | `6b55c0d7375310ea35a12f312ae95d789ce44e590945748c6825bcc8778542865d6d2e09ed1f51af7cd1786a4aa5e47edc86690dae9abd28b11c34d570f963a4` 1000 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-node-linux-arm.tar.gz) | `a06012a6c51a66f59fd3bd0675d20c167d8530998bce630fc9390c26c9b45f59acdde01f71269911d5842591a0dab186c550d1e3cb1a0a65f2d7623f6648a1a2` 1001 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-node-linux-arm64.tar.gz) | `81332cdd792efbc9caf2c3ee57567934155c84f813476305da958d5b69e200e42a10f5d2b2e495b4c66fa127709a01e343b7612bcba2c39ecaf5fe03a284a638` 1002 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-node-linux-ppc64le.tar.gz) | `fcbf2c8cbc893bed8c949005f2a9ad2ed461d30eb15d066c37ae14f35f745e701b207eaa425a8af76de54762d3046c2d68cffacaf4e9c78e82195db586c8b021` 1003 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-node-linux-s390x.tar.gz) | `8bb5852950167ee3c9402434b661e96764e915da6aa5101b99079ab2a181525ea417b64a4ea4a19313be843fdc68f42d92d4ab577ee9d2126a65c2bd37960146` 1004 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.8/kubernetes-node-windows-amd64.tar.gz) | `d22d00291b7ae6dcda480f65c1c20cec6bc430cb4bd679bcf5328c8a9d46dd9f23bcc1bbf6bf037ed3488603a050c5346b494cf153053aa2354634c0cf7aae06` 1005 1006 ## Changelog since v1.16.7 1007 1008 ## Changes by Kind 1009 1010 ### API Change 1011 1012 - Fixes a regression with clients prior to 1.15 not being able to update podIP in pod status, or podCIDR in node spec, against >= 1.16 API servers ([#88505](https://github.com/kubernetes/kubernetes/pull/88505), [@liggitt](https://github.com/liggitt)) [SIG Apps and Network] 1013 1014 ### Other (Bug, Cleanup or Flake) 1015 1016 - Add delays between goroutines for vm instance update ([#88094](https://github.com/kubernetes/kubernetes/pull/88094), [@aramase](https://github.com/aramase)) [SIG Cloud Provider] 1017 - Build: Enable kube-cross image-building on K8s Infra ([#88596](https://github.com/kubernetes/kubernetes/pull/88596), [@justaugustus](https://github.com/justaugustus)) [SIG Release and Testing] 1018 - Fix /readyz to return error immediately after a shutdown is initiated, before the --shutdown-delay-duration has elapsed. ([#88954](https://github.com/kubernetes/kubernetes/pull/88954), [@tkashem](https://github.com/tkashem)) [SIG API Machinery] 1019 - Fix a bug in kube-proxy that caused it to crash when using load balancers with a different IP family ([#87117](https://github.com/kubernetes/kubernetes/pull/87117), [@aojea](https://github.com/aojea)) [SIG Network] 1020 - Fix a regression in kubenet that prevent pods to obtain ip addresses ([#87287](https://github.com/kubernetes/kubernetes/pull/87287), [@aojea](https://github.com/aojea)) [SIG Network and Node] 1021 - Fix handling of aws-load-balancer-security-groups annotation. Security-Groups assigned with this annotation are no longer modified by kubernetes which is the expected behaviour of most users. Also no unnecessary Security-Groups are created anymore if this annotation is used. ([#88690](https://github.com/kubernetes/kubernetes/pull/88690), [@Elias481](https://github.com/Elias481)) [SIG Cloud Provider] 1022 - Fix route conflicted operations when updating multiple routes together ([#88209](https://github.com/kubernetes/kubernetes/pull/88209), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 1023 - Fix the problem where couple nodes becoming NotReady at the same time could cause master instability or even complete outage in large enough clusters. ([#88959](https://github.com/kubernetes/kubernetes/pull/88959), [@mborsz](https://github.com/mborsz)) [SIG Apps] 1024 - Fix: add remediation in azure disk attach/detach ([#88444](https://github.com/kubernetes/kubernetes/pull/88444), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 1025 - Fix: azure file mount timeout issue ([#88610](https://github.com/kubernetes/kubernetes/pull/88610), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 1026 - Fix: corrupted mount point in csi driver ([#88569](https://github.com/kubernetes/kubernetes/pull/88569), [@andyzhangx](https://github.com/andyzhangx)) [SIG Storage] 1027 - Fix: get azure disk lun timeout issue ([#88158](https://github.com/kubernetes/kubernetes/pull/88158), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 1028 - Fixed `threadSafeMap` high memory usage caused by indices that have churn of high-cardinality keys. E.g. namespaces ([#88007](https://github.com/kubernetes/kubernetes/pull/88007), [@patrickshan](https://github.com/patrickshan)) [SIG API Machinery] 1029 - Fixes issue where you can't attach more than 15 GCE Persistent Disks to c2, n2, m1, m2 machine types. ([#88602](https://github.com/kubernetes/kubernetes/pull/88602), [@yuga711](https://github.com/yuga711)) [SIG Storage] 1030 - Fixes kubelet crash in client certificate rotation cases ([#88079](https://github.com/kubernetes/kubernetes/pull/88079), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Auth and Node] 1031 - Get-kube.sh uses the gcloud's current local GCP service account for auth when the provider is GCE or GKE instead of the metadata server default ([#88383](https://github.com/kubernetes/kubernetes/pull/88383), [@BenTheElder](https://github.com/BenTheElder)) [SIG Cluster Lifecycle] 1032 - Golang/x/net has been updated to bring in fixes for CVE-2020-9283 ([#88381](https://github.com/kubernetes/kubernetes/pull/88381), [@BenTheElder](https://github.com/BenTheElder)) [SIG API Machinery, CLI, Cloud Provider, Cluster Lifecycle and Instrumentation] 1033 - Limit number of instances in a single update to GCE target pool to 1000. ([#87881](https://github.com/kubernetes/kubernetes/pull/87881), [@wojtek-t](https://github.com/wojtek-t)) [SIG Cloud Provider, Network and Scalability] 1034 - Update to use golang 1.13.8 ([#87648](https://github.com/kubernetes/kubernetes/pull/87648), [@ialidzhikov](https://github.com/ialidzhikov)) [SIG Release and Testing] 1035 1036 1037 # v1.16.7 1038 1039 [Documentation](https://docs.k8s.io) 1040 1041 ## Downloads for v1.16.7 1042 1043 filename | sha512 hash 1044 -------- | ----------- 1045 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes.tar.gz) | `dc95ba079242d2cd0b4b48f960c2b7c097e25228a2e36f08e3a611c91eedeed2d58971bf33604d28e50aae5068f04d669a8889b619d9cee5b1e71f7b5d7f2206` 1046 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-src.tar.gz) | `d8f41693601a2876f537239b43ce30cf5162d30152ced81c63d0fc01560a884abfc3c99a632285f73ab5b6292a508c00b8f0b07d6f6a5b52ce34b4b6549a70ee` 1047 1048 ### Client Binaries 1049 1050 filename | sha512 hash 1051 -------- | ----------- 1052 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-darwin-386.tar.gz) | `d80cdde0ee493da3e7780a2468031554bea1c6e9f078e2b15c2fc688472d1ba591ded2653b5c011ec6b11e5671194a166f83ae375d192df7b59437e6c3c3a09a` 1053 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-darwin-amd64.tar.gz) | `6379aa1a10dd6922ca4ef4e03d34c0ab02a7c955052f533c2023b34f9b79f27baec0adcee0f0da83e034de019f4c0c66738e4879505016dc47a70a57b44b100e` 1054 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-linux-386.tar.gz) | `07f33999629f88dbad5dad1cf56671795d003506ab2c8295ff77ad1470d887f551927dd201a625dafef3f93fb283f6b7366509c050bdfe16cdd45d779c61f7ac` 1055 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-linux-amd64.tar.gz) | `df4b8ef210f5daf5b75000f2ebc54c3e10d0f30da97adcd83eeef7bd3f193df440c47da73bc71e83dae8d52a3ea7251d2b3293c84594da36187d193a5d26576f` 1056 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-linux-arm.tar.gz) | `d33f094701da1c35c83459648beba90bf525375b82831e34a45da46ac11013b42ba3aa8270a9e3af14f6afae980b298383759e477a492980f868cc18a9510f1b` 1057 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-linux-arm64.tar.gz) | `9024da038c0ebfb739787e75ca2a78fff0265d4b0bb241648f76ff18f0bdd731da6617182d0f0d827668d81c27f24e95dd26102f5e297ba46a2fbd3dfb2d05d9` 1058 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-linux-ppc64le.tar.gz) | `362e67bf6f9d8c8dff2c7e8ef0fab6701992575c9afe640edccbe86426dfe98864be663217b7354e1ae1701c505786256b740a98fed2c08338e0caa132bb1faa` 1059 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-linux-s390x.tar.gz) | `958f1d9a98e2b75aebb3a5e33882205a0935f8cf812dbcd152242a295a5534b592c3ffada169f0ed183ab32f87929f5c706aa1db0ab1ec4be2d88a3819480d8e` 1060 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-windows-386.tar.gz) | `87e58926ffabe47b42dedcecf34212a92f67326826b6928cf8143f0baf03cc71cd4b2bcd0f01b9947742bac4ab4cbd24c5c0770aabc4be38e36481d2dd280bb1` 1061 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-client-windows-amd64.tar.gz) | `c7e7892b7845246670bb2d309c55196e4635e10f5c651de281a1f8045f122a70194a397e4ee900e1a0cd65761d6a03b76843ca8c16ddc89420df28a20bc9b999` 1062 1063 ### Server Binaries 1064 1065 filename | sha512 hash 1066 -------- | ----------- 1067 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-server-linux-amd64.tar.gz) | `e49a47530efc893118e0042e0f0ee3d04f01240bddf7f4e1e440d0da27f822a51e9504c08cfd7b28c0d82dc43252f64ed46c9eb7f31f7c8da88098705174a16e` 1068 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-server-linux-arm.tar.gz) | `7b5f3f9a82d2864bd155339ecdc260aaa09bd0d95a594599583f57dade5ad43238b77a8d1c83a46e25a899693ce2dcc07fddbefab4ef6c3ec6266cad126dca41` 1069 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-server-linux-arm64.tar.gz) | `66a2d2b49045a38aa6a1797aba3cae676aee5e21816096f3853efe2ab050575739e01c361bc8377f2af065f6e41e8ecdf99e73fb5cf619263778aef31fd630f6` 1070 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-server-linux-ppc64le.tar.gz) | `ff8c651bce2bbd271e90f3964cb5e0674dd20054c451ccb9f542c4f8b7b75ad71f477981c6d59135d424b5ce0f69484fa9376398de9837a7cf34d58fb4a94bf8` 1071 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-server-linux-s390x.tar.gz) | `5836ee2774de947ebaa5ae08a8a86bc0213c7c3518de2461a4b5aa7e4a60d49556c27bacada22b848522a937cb9f6765272dacb2c664d4adf46ea7e8a51fd527` 1072 1073 ### Node Binaries 1074 1075 filename | sha512 hash 1076 -------- | ----------- 1077 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-node-linux-amd64.tar.gz) | `1bc7b5eac5e36b3215356e8d2447cfd7061571dfc0a8a87d9d021da0b38e904ac2a6a8a9652adea619063aa09e6b74a014cbeb20ec67a62ced81917392dfeac4` 1078 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-node-linux-arm.tar.gz) | `4f0950d43478c8f94030fd25a30c21d9d9555465fc7183aa4325564dbd223c21038aaa02becd629ac3990bc5a4d0b7fcdec386b9dc8b60efd7d5041be6c12fa2` 1079 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-node-linux-arm64.tar.gz) | `f272c4ad555ba8d5d4fbb5650edf11c1ee0cd5bdee53df06a494108661be450d101a4c9cd4df16709d88d36287e6bdd8aa4431b7d87c79e3efddd3a8ecb43eb1` 1080 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-node-linux-ppc64le.tar.gz) | `2b0c908e2167c919b2e7e85df7160de7815949e95f4fef00b767cb931924ad4119294dc8784aed2531db2dc3ed99942fb1efa03cf4b844ad896111f50a23c06f` 1081 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-node-linux-s390x.tar.gz) | `295e451c9e278ebb8b6fa15f5df43ab127f7acf02fcaba609de2ab4b36f12077fe74716773f4e23f1acbba8a14619ed84a407834b1ee8694f0af95f5b0cf800d` 1082 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.7/kubernetes-node-windows-amd64.tar.gz) | `53af7e2ffb41a7f87160128326657d8a73549ffb22c7b6d17a4283f8f574bf60fd5c3b2a2d8ca92058b0943be3d1323f668329b3fac3b7a53ab853298c0f17b8` 1083 1084 ## Changelog since v1.16.6 1085 1086 ## Changes by Kind 1087 1088 ### Other (Bug, Cleanup or Flake) 1089 1090 - Bind metrics-server containers to linux nodes to avoid Windows scheduling on kubernetes cluster includes linux nodes and windows nodes ([#87022](https://github.com/kubernetes/kubernetes/pull/87022), [@wawa0210](https://github.com/wawa0210)) [SIG Cluster Lifecycle] 1091 - Fix the bug PIP's DNS is deleted if no DNS label service annotation isn't set. ([#87311](https://github.com/kubernetes/kubernetes/pull/87311), [@nilo19](https://github.com/nilo19)) [SIG Cloud Provider] 1092 - Fix: set nil cache entry based on old cache ([#87592](https://github.com/kubernetes/kubernetes/pull/87592), [@aramase](https://github.com/aramase)) [SIG Cloud Provider] 1093 - Fixed a bug which could prevent a provider ID from ever being set for node if an error occurred determining the provider ID when the node was added. ([#87043](https://github.com/kubernetes/kubernetes/pull/87043), [@zjs](https://github.com/zjs)) [SIG Apps and Cloud Provider] 1094 - Fixed the following 1095 - AWS Cloud Provider attempts to delete LoadBalancer security group it didn’t provision 1096 - AWS Cloud Provider creates default LoadBalancer security group even if annotation [service.beta.kubernetes.io/aws-load-balancer-security-groups] is present ([#87207](https://github.com/kubernetes/kubernetes/pull/87207), [@bhagwat070919](https://github.com/bhagwat070919)) [SIG Cloud Provider] 1097 - Fixed two scheduler metrics (pending_pods and schedule_attempts_total) not being recorded ([#87692](https://github.com/kubernetes/kubernetes/pull/87692), [@everpeace](https://github.com/everpeace)) [SIG Scheduling] 1098 - Kubelet metrics have been changed to buckets. 1099 For example the exec/{podNamespace}/{podID}/{containerName} is now just exec. ([#87913](https://github.com/kubernetes/kubernetes/pull/87913), [@cheftako](https://github.com/cheftako)) [SIG Node] 1100 - Openstack: Do not delete managed LB in case of security group reconciliation errors ([#82264](https://github.com/kubernetes/kubernetes/pull/82264), [@multi-io](https://github.com/multi-io)) [SIG Cloud Provider] 1101 - Reverted a kubectl azure auth module change where oidc claim spn: prefix was omitted resulting a breaking behavior with existing Azure AD OIDC enabled api-server ([#87507](https://github.com/kubernetes/kubernetes/pull/87507), [@weinong](https://github.com/weinong)) [SIG API Machinery, Auth and Cloud Provider] 1102 - The client label for apiserver_request_count and apiserver_request_total now no-opts and merely records an empty string. ([#87673](https://github.com/kubernetes/kubernetes/pull/87673), [@logicalhan](https://github.com/logicalhan)) [SIG API Machinery, Instrumentation and Scalability] 1103 - The docker container runtime now enforces a 220 second timeout on container network operations. ([#71653](https://github.com/kubernetes/kubernetes/pull/71653), [@liucimin](https://github.com/liucimin)) [SIG Network and Node] 1104 1105 1106 <!-- NEW RELEASE NOTES ENTRY --> 1107 1108 1109 # v1.16.6 1110 1111 [Documentation](https://docs.k8s.io) 1112 1113 ## Downloads for v1.16.6 1114 1115 1116 filename | sha512 hash 1117 -------- | ----------- 1118 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes.tar.gz) | `cde4a4285400f92a47ced2c0bced384408b2e38c429726ea8a60ea63bd0bb6efb67282c424409ba92c8e97334708820a21b3b639364c6b2184ae4d5a3a254f82` 1119 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-src.tar.gz) | `1d534f5f014e3a0d876932ecf932000622c8d435c056e553630898c017fd19cfd9408ad887a61c252f232bbbd5d7535b2a0f8dccda362ca1468a5ef1a3b40602` 1120 1121 ### Client Binaries 1122 1123 filename | sha512 hash 1124 -------- | ----------- 1125 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-darwin-386.tar.gz) | `6f83f4a85110d778120da77c4ae9bbbd2ad3cb39ff2bdeb4966452fdd8e27f73f287a7d2a7aff82dcf45de973d655cc026cb874a3cd63dbbd3a939db12327a74` 1126 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-darwin-amd64.tar.gz) | `7708012605e73ee16edb9124a714571ef943a702eb8e7e20a63932d75a56bfcde46020f0babada681cb2858bb4dfa30749c6b00534a6a70e0d682634507887a4` 1127 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-linux-386.tar.gz) | `9de5684263db702f42a38ff770f35601118320edf8e9d48887b24d55af84571e7e13baf68e2a64ec2a349f740a9d8dedee882b144ed46dd86a8fb33993887c01` 1128 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-linux-amd64.tar.gz) | `fa078c6e66c7fc7ec5a7551c714b14b44b68817d87d8a4238a17f1452a382cfc6455a2c3468ce45ba4edf473036af7b92668ce1ba6df05c5dbd902f6f308d5f9` 1129 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-linux-arm.tar.gz) | `7f0ac2703407e159fee2159056127a45e9c4538763ebb69b5085e3edece22b31d0fdf2b7b1d3febc2d495c7e187033638d8f721e052f510dbd4c1a18435f5764` 1130 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-linux-arm64.tar.gz) | `ea83f7c5b6376437d1c5d87dc833e78b4931b4b3b1fc2a4e0a0076483a850c1cb48099ab53f414ab0e38c206ead4cbbfb08c45871b928823b77a6d90fe72867c` 1131 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-linux-ppc64le.tar.gz) | `29038b137eb9e00767e3faeb2d72f45c3495c273c88541330ad870089ec868c96ab7ce826c059858760500a4bbc0623ffa434b42ff528f1984eafa3223e5bd2d` 1132 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-linux-s390x.tar.gz) | `f7a31fccf23c3ba114efb087facf2ef729a3c5288d0e53da550a25c0756554e7740e6ba4d058c7e0098b5cebc42298ad3d8237b94c395f2adb37c8de552c8c07` 1133 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-windows-386.tar.gz) | `de14a4a163bcd6b38edadb22a440d9fcd70d69ac4fa81b11d00c860d8bfda4698be588f3823234dbae7ea6195e7968b510065f2111d70e274fdf667a5aa7a648` 1134 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-client-windows-amd64.tar.gz) | `c209002db706691e3a44bdf65ed99d36878c0fc59df0121c74a4e32f35a701bdd30fe3c5ed3424b1e85c94b7df41ec657e93f60d2c9b5ce9115250e18f172d42` 1135 1136 ### Server Binaries 1137 1138 filename | sha512 hash 1139 -------- | ----------- 1140 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-server-linux-amd64.tar.gz) | `eb647feb475550c3bb3841a91a1512b0836d0cee9eef0cda3292fe182307caa27be8a589dcfae8ad7cd70d0b028ea3645a36e7d486a03bcb84d351d08c5e79b6` 1141 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-server-linux-arm.tar.gz) | `cd1994e92768b5f879ff856ccb6f688646d2f6011c543192b1ece16a25e585b2baece4bd74341d95daa2c78d60fe6c0b0d96e8ea8691bd109529290770438115` 1142 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-server-linux-arm64.tar.gz) | `6119311e77d194b3daa81d71fe3e2bf8762ea9c48417215b9a52d2b351c45751b4d4222afb21ea33f205451c6e183ccb224bf60b939a085f9dcb7c321e855407` 1143 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-server-linux-ppc64le.tar.gz) | `5f6ff761146ec59cf118caf36c89545b5ff3c6a3536a5c7a275da365d9dbf420a80af45e809830f4d27e72b2d94b62206c6e3c229152b4dfe80eb205f2278b7f` 1144 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-server-linux-s390x.tar.gz) | `7d35c984bfbd520078785d2164d5a96e27953536adad1f7b0f39f3f6fe82461dcce8f0fec5fc13720446a9a4380288ae7cee523632aa8e2dd2e4a053c8aeb3b2` 1145 1146 ### Node Binaries 1147 1148 filename | sha512 hash 1149 -------- | ----------- 1150 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-node-linux-amd64.tar.gz) | `a39c7ab21f58f7d47ec17a8ecdb6f08a3f6302ea6dcdd54fee87660df4325cc3557055a60da190f545b1c763140eceaa44bfa5ee86c24a8062a5652bb441e62b` 1151 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-node-linux-arm.tar.gz) | `ab7ea3164f83a719a6dabf701e6673b5c082fe9b26a111fb354eb772a355d077c5ff992331683b22b970706d4badc2a35bdd32b4c1a17e9fcb0a40f48925aaab` 1152 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-node-linux-arm64.tar.gz) | `5c4f788f37d48fedf95ea8ed9e5778219c2847e40e63de06b5342aa8564321bdcf3dc224cc9bfe13526a31ee3d1353230abdbe59bc0119e5f231a48e8264df17` 1153 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-node-linux-ppc64le.tar.gz) | `2a67a8c2b0455077aa35a8b949854a27a9ce7c36f169a75ecec638fba9115eef4949f6c6be9f0dd3f8dc1485b47dc5ab5938d77391b921edfc224d9e774e770b` 1154 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-node-linux-s390x.tar.gz) | `cced562021cf512967fe2c3ee169d6f20c00df242b47d4a3d7c8dce47da3a622bcb8f50cf10fa92e6b9a1e2f345487d58f1926188cab9df09142fdf26078a88f` 1155 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.6/kubernetes-node-windows-amd64.tar.gz) | `cb0ca5f0b7e051409d8da99c8d8270d4fb7565e85f963dc55d8874f9d686f9050ffedeb84a0b82a26921ab75ff87d854c7b6849231f34c8cdbae275cfec15189` 1156 1157 ## Changelog since v1.16.5 1158 1159 **No notable changes for this release** 1160 1161 1162 1163 # v1.16.5 1164 1165 [Documentation](https://docs.k8s.io) 1166 1167 ## Downloads for v1.16.5 1168 1169 1170 filename | sha512 hash 1171 -------- | ----------- 1172 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes.tar.gz) | `574c4abd42d3640eb98bf4144afc64bbe25b752235d3398e3717a64fa505b5848e3e0b7a31cbea13d936201844c4f6385006556cf8c191f7bdac638a388345b9` 1173 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-src.tar.gz) | `b85014dd97af562f966505cfaf7346aea61ed4a027d58655dc0c6148f2893e8f4309465fc0f66e3abc1e394c34f64754df78229384aacf529e834b026b84f96b` 1174 1175 ### Client Binaries 1176 1177 filename | sha512 hash 1178 -------- | ----------- 1179 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-darwin-386.tar.gz) | `becf76f632c6e1cb65074e0a0f9b4d978d9f2c6896faca29a46d4059d4ee5d8663c44dfc868157a1549c333c010015cb91fa1299c224c38af2ee49d84801bfa2` 1180 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-darwin-amd64.tar.gz) | `b073fbe7c435b8498e1a154ba4c50aec7bc75956c3ba3d9679ecb3e172716aabf3e6190309ee508449efe0bf01eeec0bccf4620d0cbd73323e6af9ba09e65cb3` 1181 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-linux-386.tar.gz) | `e6462531b8e2df8e04aae1e75bcf78cdc5879ff90d09438355388d14cbee5b0d1895c1f9cfb8acfe50cb0aa246d8a900050b52b77383e76497417d810f7bf7d4` 1182 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-linux-amd64.tar.gz) | `d37424d8d5a275aa949d9b92d448643e4951ae16458135f44f2f4e3c66f379a4fe53e4d46bb0abd07623bf7b8b7bfd7bf9de019a3077dc28e26d13b0533c5559` 1183 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-linux-arm.tar.gz) | `9b47ed83fcf48355b7ed53619aa6cb32401e367de839f71d316b9ce52000180a358547bf03b58e1cfb763c5bafb6c8841c1b4bfbc6ab8428a51024323549270d` 1184 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-linux-arm64.tar.gz) | `762c8a4f65b370c4707770cb211014d4fe2e6f0a19eab883bb06a36a29cd8db3e30a044a11d327d11140b4fb1f555849730c71b08b469f9edc13def79df0cad3` 1185 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-linux-ppc64le.tar.gz) | `a84010df85b0bb4e8f3e75ad2566fe1e5378dcb812cbd2888c9c5f81640db849b0f82f5ebac3d585e67c37b2c6ec74d425870a66a89e399a64cf59d414afbc6c` 1186 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-linux-s390x.tar.gz) | `268dcae0bea43cc91f7eaca7695ffa037875de9f0dfa723667dad24dca569083a609f8873e4c4491ecbdba902b61ec3c39a223ca5e5c4455142109fc5c9c428d` 1187 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-windows-386.tar.gz) | `90895ca4a375fbb4f638d29cd5fb6ea7a49bbb8a6d608fbf4474e99a5b82903a0b7a71fb7234d9abe410100616072ac5560bf5a6698bd41cb23e7c0fa5beec88` 1188 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-client-windows-amd64.tar.gz) | `df6655957b8f8aa0e2baae25b7018753a9f921b4b08a509d9ac22fb299151fe04e5aded9196b7c5b0e5d42a02791d7e973266825fa938479746052f2fa1a7c02` 1189 1190 ### Server Binaries 1191 1192 filename | sha512 hash 1193 -------- | ----------- 1194 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-server-linux-amd64.tar.gz) | `673308a4a5e4f01aa45d44f7fdbb4c66f7158f47e74cd0f8dd2b0877f18dee71b71973ca94e2a8c075014a14542c680d4b0c43cd9794245965577e929ee0b736` 1195 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-server-linux-arm.tar.gz) | `646cf65bae430d78e201c3c4a5c1fbdbc5361cc07e4271b3d04f28d5838060a7f5e47f1bafaeac26862d1a484ec6d2f5c4ae1163609ccf0c4b2a640ebcb7198e` 1196 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-server-linux-arm64.tar.gz) | `c0fd58e4fb4fb923a5d9f421fb3f574940c99e71343bc7bfa06b24be45b4f4053fc6fe1e6ad1bc6585dfd54a4350864edbec75f376dbb5609fb158da122676ad` 1197 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-server-linux-ppc64le.tar.gz) | `3ea6a69d4ca2254f0e4df61e57de9cffdd0e5fcd1fbba5b3772e969f54f1399847c3f580a3f7573d7a720d066122c504f8093ca0b8fcb61b2e7040dd7b99d174` 1198 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-server-linux-s390x.tar.gz) | `a00cc44fa9a3ad320573cff4142a9edb80e7c1305827ba81a4c6bbaf92870215d05b63280de721d3d8015bc721d2e5cbef67e23ba949d36c361cb4e519d6829e` 1199 1200 ### Node Binaries 1201 1202 filename | sha512 hash 1203 -------- | ----------- 1204 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-node-linux-amd64.tar.gz) | `5ea212cb4cb06f099639d5a573e3ac3c4df38d12a76695298abffaedae6a7c0259b974e7b185f67fca39791cedef7e827a908e0c2e0f6ead87010ab8d7bd0fe9` 1205 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-node-linux-arm.tar.gz) | `173c7aa6268747f594d8edf355fd392dcc29b8685c002f54e8577fd6797cc8bf8e4599e728eb40b9d41239b420596cb1acc79a4e6430d1d51f62b002753e15b7` 1206 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-node-linux-arm64.tar.gz) | `d302eee8cf5b851db3b3d7ef5ae1e1f2f8b930d44d03df94507d54ba04f0a9f1960a387db17014e815043f77ed66477037b4602896030bbb2b7421be000db981` 1207 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-node-linux-ppc64le.tar.gz) | `284afd70e3bd37bf4dc1967ea04f7584e1f9387282416940f5ab720f626e070dc0a5383c90035a1280294dc2e4097650f896f2dc40c47954795348ca9db8d173` 1208 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-node-linux-s390x.tar.gz) | `a600f3872c8a4a8207e384792286da96f3fee58ba488fa3491901d90e3a67abe2d1c2f3642b1551593e0fa7364a01b10aa25b49bddfd97df8ac642583842675f` 1209 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.5/kubernetes-node-windows-amd64.tar.gz) | `fc8267a1ad5f470b6a630f2b6c1bf812cd41d86e7cc77f938f07d7cb88ec795b24b271f3b49cffe9762fe7a507fdcdd7b59ca36afb40466bbf44d9bda01d1cf0` 1210 1211 ## Changelog since v1.16.4 1212 1213 ### Other notable changes 1214 1215 * Fixed a regression where the kubelet would fail to update the ready status of pods. ([#86191](https://github.com/kubernetes/kubernetes/pull/86191), [@tedyu](https://github.com/tedyu)) 1216 * Fix nil pointer dereference in azure cloud provider ([#85975](https://github.com/kubernetes/kubernetes/pull/85975), [@ldx](https://github.com/ldx)) 1217 * fix: azure disk could not mounted on Standard_DC4s/DC2s instances ([#86612](https://github.com/kubernetes/kubernetes/pull/86612), [@andyzhangx](https://github.com/andyzhangx)) 1218 * Fixes issue where AAD token obtained by kubectl is incompatible with on-behalf-of flow and oidc. ([#86412](https://github.com/kubernetes/kubernetes/pull/86412), [@weinong](https://github.com/weinong)) 1219 * The audience claim before this fix has "spn:" prefix. After this fix, "spn:" prefix is omitted. 1220 * Fixes an issue with kubelet-reported pod status on deleted/recreated pods. ([#86320](https://github.com/kubernetes/kubernetes/pull/86320), [@liggitt](https://github.com/liggitt)) 1221 * Kubernetes 1.16.5+ now requires go1.13.4+ to build ([#85019](https://github.com/kubernetes/kubernetes/pull/85019), [@liggitt](https://github.com/liggitt)) 1222 * Fix LoadBalancer rule checking so that no unexpected LoadBalancer updates are made ([#85990](https://github.com/kubernetes/kubernetes/pull/85990), [@feiskyer](https://github.com/feiskyer)) 1223 * cherry pick of [#85885](https://github.com/kubernetes/kubernetes/pull/85885): Provider/Azure: Add cache for VMSS. ([#86049](https://github.com/kubernetes/kubernetes/pull/86049), [@nilo19](https://github.com/nilo19)) 1224 * Fixed a bug in the single-numa-policy of the TopologyManager. Previously, best-effort pods would result in a terminated state with a TopologyAffinity error. Now they will run as expected. ([#83777](https://github.com/kubernetes/kubernetes/pull/83777), [@lmdaly](https://github.com/lmdaly)) 1225 1226 1227 1228 # v1.16.4 1229 1230 [Documentation](https://docs.k8s.io) 1231 1232 ## Downloads for v1.16.4 1233 1234 1235 filename | sha512 hash 1236 -------- | ----------- 1237 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes.tar.gz) | `c81e0e69b81b2dcd3d2397bc54756d77a8d41b88adba12eb5a1ffe71672554ecb1ff97c3979d0a81a218477440308860f9076124d6c0a3f0670ea21759a6621b` 1238 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-src.tar.gz) | `b27c3946a6b466e5c1c0f1ce25d6af037ee0842d52be424d75ccfc10f9c7b6acc061b03d327ddd1889c0ec11dd46ea0872eed8d428013ae1e69d712b32f78c2e` 1239 1240 ### Client Binaries 1241 1242 filename | sha512 hash 1243 -------- | ----------- 1244 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-darwin-386.tar.gz) | `26d6708b788a8524513897133199b7c0737d0578bdf944692c9393f635fe8f0a202a147eabf33748ec7e849045f5a0a19ae0ff1b04b1de7d565dd7fdec797a08` 1245 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-darwin-amd64.tar.gz) | `d4efd604d166f12bd860888dfe379490b64e05ff615855e45314ef405a8d5fa6dc51c1da3abe20b15b155dc7742ac2b929d32c66c3de3e5b497b18685f5fb9e2` 1246 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-linux-386.tar.gz) | `8366ed646ab3a387341455e3059ea02f5238e515303969155d48d735e4af9e8cc47eb9eddca370c854cbe599c4615a77c46dd0be1f893dd252dc1d3246c6304c` 1247 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-linux-amd64.tar.gz) | `30aa7aab0f82b1ad20f53ed693666bf39a84c3891549c58988de8e0106f4e5f46725f78de0d149ecdbb6e6f48d8ee7891d2b64c81a9a13cdae41ca6b147500b5` 1248 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-linux-arm.tar.gz) | `61eb1750800f07b4f8c41da9f87e3fed5c6a42543c46031087ddf334aab3baa413433f66cc3314b62ab23aba594811dc1e363b2b56d9c0c4413135f33b4e1343` 1249 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-linux-arm64.tar.gz) | `ffce76381f999ccd51383321824404ee240d69374720077fcd2fa5b0b523584f1d4504538377f3c9a07ee4ad38ad2bdaf5ba1b80c1dbe3998f07a386ea26ca5b` 1250 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-linux-ppc64le.tar.gz) | `c76d6250028122e2e63ca2ba447b989d4bb476c0eddfb0d0152d33a11e00bf189fd954456fb6d4508505523d8d13f2218878aafba62b114244cd10d328a88a0a` 1251 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-linux-s390x.tar.gz) | `cf293f9c8c02201e143aeb3eb6157621156883057b187d0cabc0ec196b7690eeac387dc306053b14c78657cd880a7e775bd966395102bba6cb8580a717b49bb7` 1252 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-windows-386.tar.gz) | `189bd08bb875d5b98bdcdc45b0ae498f52bc438bc59c45080e280349ca92f77a1b05d0677d52421b89f4d1123585f6c9618f97016eaad1d64eb4ede6922facfa` 1253 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-client-windows-amd64.tar.gz) | `ce437278896a75d0f5d13f318b59b6f48f39dd04cf70ca982fe5fe9edf4da23691b3993e5dd8fde1acd7aff8d599ac27d78887fd1253b80a9b65c14f7833eac3` 1254 1255 ### Server Binaries 1256 1257 filename | sha512 hash 1258 -------- | ----------- 1259 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-server-linux-amd64.tar.gz) | `ed3a3130c34aff712d8591ac2119fe4f6a8ea430afd1d88fb987dbd6f4e61f1a1d734d8491348ee4fd71ca04f2592accbd9d2666d2c816949c8111dcd9e99233` 1260 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-server-linux-arm.tar.gz) | `60d537ae2a94f7e3573ee5085eb14ef715a53b4c71516dbbdefc483a809b16fec26d8a112382a35bb4b26ef8c11ed985529d1d7dcc50a4526ec9ffdcd43648b0` 1261 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-server-linux-arm64.tar.gz) | `afb0f2ac7bd92ba107d9cffc9d3723a732d3c51d8b0956af7cc243dc74bee496f7995de75fd9cc758b2b4cd498db8552c3a7ac5c44a6c57fd97a6728a37b4a85` 1262 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-server-linux-ppc64le.tar.gz) | `54a9d9c48259718fbe1b49cdd6ea699b8542633bd2ec676ee13e725104064ab7525aefa712b283177dbfc2c71bda2348344fa229c737b1ce35cfdf0aa6c6dcc1` 1263 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-server-linux-s390x.tar.gz) | `15525320b414ab13975c08fefa3272928615f8c3898780103fb33f6ade4342e478a566f31b876b78484fb06e39a2ccfbc9986f781642414b7760fc295f1eb10b` 1264 1265 ### Node Binaries 1266 1267 filename | sha512 hash 1268 -------- | ----------- 1269 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-node-linux-amd64.tar.gz) | `c25fa6dcd8a62338a4fa748ae6be4c6501733d6012fffd73d61f2d7c9322d6fa8fef18c149f6ea9232bd65cc4d31f8500d25e6c9dc36fd8b0606f934bc5a7d40` 1270 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-node-linux-arm.tar.gz) | `3093693fc1af4e319d0fe17700e5ad79a0780db03893e201188e7c5f4ad501761b90ebec251eb63c5a2355ea1d922e04172f86bebd560b07f20d39a72b4fea70` 1271 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-node-linux-arm64.tar.gz) | `93d96b1cd33ada44c8d2f97775edc2782c042d88389b556deec04e8876cc32a726ead5ef4ed3342fa1af2cb23927c3fa43af440d685c340146de6f122c4d5f03` 1272 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-node-linux-ppc64le.tar.gz) | `7b607c0afd4cc373d57efdbe88b0e82d087c790d35bd1b98fe12b0b9321dd10d8189cdf9b3c5cf36b674aaeefb2bc6234dde2f83d9adfb8a9c8160fb26335d86` 1273 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-node-linux-s390x.tar.gz) | `43120a2a3b741ad2f40e17156963a5cb0af49190e820ce75808fa1c3969e1dfc96d509a591c328fd6c037e2b4f6380bfd6f0dad5c2675eaa92d635cfe3f74c49` 1274 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.4/kubernetes-node-windows-amd64.tar.gz) | `fc769cdcbc5eeac8ad6776864a21bc75d2d1cfc682d0c0c51128b3cfcadf8cb14b815c29e545491028c99c35c6ab5a2a4dbfb9e462a92c5d1bc2ab98f8c70a65` 1275 1276 ## Changelog since v1.16.3 1277 1278 ### Other notable changes 1279 1280 * Fixed issue with addon-resizer using deprecated extensions APIs ([#85793](https://github.com/kubernetes/kubernetes/pull/85793), [@bskiba](https://github.com/bskiba)) 1281 * Fixes a performance issue when using server-side apply with objects with very large atomic maps. ([#85462](https://github.com/kubernetes/kubernetes/pull/85462), [@jennybuckley](https://github.com/jennybuckley)) 1282 * azure: update disk lock logic per vm during attach/detach to allow concurrent updates for different nodes. ([#85115](https://github.com/kubernetes/kubernetes/pull/85115), [@aramase](https://github.com/aramase)) 1283 * Filter published OpenAPI schema by making nullable, required fields non-required in order to avoid kubectl to wrongly reject null values. ([#85733](https://github.com/kubernetes/kubernetes/pull/85733), [@sttts](https://github.com/sttts)) 1284 * For x-kubernetes-list-type=set a scalar or atomic item type is now required, as documented. Persisted, invalid data is tolerated. ([#85385](https://github.com/kubernetes/kubernetes/pull/85385), [@sttts](https://github.com/sttts)) 1285 * Fixes a bug in kubeadm that caused init and join to hang indefinitely in specific conditions. ([#85156](https://github.com/kubernetes/kubernetes/pull/85156), [@chuckha](https://github.com/chuckha)) 1286 * Ensure health probes are created for local traffic policy UDP services on Azure ([#84802](https://github.com/kubernetes/kubernetes/pull/84802), [@feiskyer](https://github.com/feiskyer)) 1287 * fix vmss dirty cache issue in disk attach/detach on vmss node ([#85158](https://github.com/kubernetes/kubernetes/pull/85158), [@andyzhangx](https://github.com/andyzhangx)) 1288 * fix race condition when attach/delete azure disk in same time ([#84917](https://github.com/kubernetes/kubernetes/pull/84917), [@andyzhangx](https://github.com/andyzhangx)) 1289 * set config.BindAddress to IPv4 address "127.0.0.1" if not specified ([#83822](https://github.com/kubernetes/kubernetes/pull/83822), [@zouyee](https://github.com/zouyee)) 1290 * Change GCP ILB firewall names to contain the "k8s-fw-" prefix like the rest of the firewall rules. This is needed for consistency and also for other components to identify the firewall rule as k8s/service-controller managed. ([#84622](https://github.com/kubernetes/kubernetes/pull/84622), [@prameshj](https://github.com/prameshj)) 1291 * kube-controller-manager: Fixes bug setting headless service labels on endpoints ([#85361](https://github.com/kubernetes/kubernetes/pull/85361), [@liggitt](https://github.com/liggitt)) 1292 1293 1294 1295 # v1.16.3 1296 1297 [Documentation](https://docs.k8s.io) 1298 1299 ## Downloads for v1.16.3 1300 1301 1302 filename | sha512 hash 1303 -------- | ----------- 1304 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes.tar.gz) | `17618d2ece64346d03db6a6c31eb3c38bbc3aa206005e99032791bdbe7dad1f401bdb9d707995ff83b658dd1ad59d9b53489e5ee45b9f209edf09b35326aac4a` 1305 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-src.tar.gz) | `e51d3418b006fb28039dc30095333f9e362c785eb27778554005a83e70860b33b554f61ddb6dd83e05d1ca0dda4c6f9bf2812347109e83ff72ad4d355bd22546` 1306 1307 ### Client Binaries 1308 1309 filename | sha512 hash 1310 -------- | ----------- 1311 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-darwin-386.tar.gz) | `15d3316afc04ae959658cc4489347b1e425df37ce7438e79357bd7fde783766374315c4c6e4ea8a94dab8d113c1871aff70de84e3181d08c37849c4971f6ad7e` 1312 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-darwin-amd64.tar.gz) | `82c2af675d77dea335d99247df9136668bda81d64da82a949543369bb31e0bc2963f9de83cb34174b01a93f6a1acab3459fb64621e771e4f7362c090ac2a256d` 1313 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-linux-386.tar.gz) | `193ae911e58fa61e001730cfa8018cb77b672b77cba6b3916bde53a73add160641085176d8bc60ea25e22f7b9a8d2a9ec17da139ec234771939cc55d5b2c0931` 1314 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-linux-amd64.tar.gz) | `904604839bbf46c11c7934f6f906adbc968234044b9f3268b71c175241626f8d4076812aca789dc5aa2f85fd25a384ad779638231d4d94f3f3c6d043b5d9f062` 1315 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-linux-arm.tar.gz) | `016e9a5662afa27da5ffb055fbf2bf4135b8521deb6bd3dea565f250f611f1690be9d6ea18c3fd1053c71c03213bc53441d88e70633192a443b838d353783278` 1316 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-linux-arm64.tar.gz) | `2ee032304114556bf3ab78c000f32ab9605e294030d83edbe013fdeb19eb49283d8b8a9c120e2d076f394f2e0d74015384754e6876e62f68ec515378ddbce77e` 1317 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-linux-ppc64le.tar.gz) | `07299344bc4c2da573b2bc4b2fa9c7a60358c4a591d711f8a9ce45768543fe678d1f15412db0fa02b777964cebcee2bcf9dea16d94a97754eac163b77cb6ddc8` 1318 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-linux-s390x.tar.gz) | `83e6a17b86740199788b48e94534ea6cd08c9325325a7f75a7073771a1de421843c2898ef385a9d8912d7cfd99c09b60a4bb4f3c6220c8f212c08a72a85661b2` 1319 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-windows-386.tar.gz) | `bfc6ff8fcc7e1219074df6db5d42fb38af4b6c10e36fb0fd7c6d8aa72fc01cbca5658d8c6a9882ed234cc20652511f551c5f59d77a0a46fc2b27f96e6f4e385a` 1320 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-client-windows-amd64.tar.gz) | `ccd9d4f53152f1c09cb5c2194892afaf93faf7bfbbf0b8ea0d6c333f121593b7c7d0039246af9234b684ac8501e4f778de8ddf02d4b34ca191621f7c74c811ba` 1321 1322 ### Server Binaries 1323 1324 filename | sha512 hash 1325 -------- | ----------- 1326 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-server-linux-amd64.tar.gz) | `040f6bfb78cb336d2d23360c0921ef7e008561b2f10bed3261c33811d1d818693cd24dcb5bf52cbf854f4209a8968a5e07ff8356e3e061a4db14623469098e5f` 1327 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-server-linux-arm.tar.gz) | `e8d8312f44ecd38d8d88300cda22ef364a637af4a666869e0926bb25008b31eceb5cbdc4ba2fda9358d996c1b9776fe4ae49693b3e95237bd82c2ec7898b4249` 1328 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-server-linux-arm64.tar.gz) | `39e7d76e88da1ac712faf6d61bb980c0a4ac01d4996bb559cc6a9c132105a5b073d7833d431ba44418b4981a190ff613463cbf16f25e4ce38a31f43191842680` 1329 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-server-linux-ppc64le.tar.gz) | `72c418ade3291911da4a654ae4c4a88f7d02ad1608d9d9721e9ce7199c4dc07090c9f53b6350911050f25c7e583423ba9e985260e94982e217119b2b0a7be501` 1330 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-server-linux-s390x.tar.gz) | `7405ca92726fb8e4083ee66a996df98e37c1a04b679634fadb0f2e64aa7e264a5e55b1149fede58d7dc693c71551e31abbc1e2bd8b137686debd2091ce9059b4` 1331 1332 ### Node Binaries 1333 1334 filename | sha512 hash 1335 -------- | ----------- 1336 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-node-linux-amd64.tar.gz) | `6074a03879f385fb3411b024ada7e1d2b8ae89990fe4f5a9c6046424cbb567afaf24264a003157b682c91eedcbd09bca75a51265fbdb3a948930fe7494c997e2` 1337 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-node-linux-arm.tar.gz) | `277432b212c98e7eb8020b17745d4c90dbc6c359148efc38e41e0836085eac4655effd9e5a694162d61ee31adc8eb1e9f6196fa9f05d898e2f4d596a1e097055` 1338 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-node-linux-arm64.tar.gz) | `6354e8862e60d6c6d60a115dd6043beed8205991c833716426071dc70d3658da1ddc9b23c4e16cae0410a86ea82eab3240f2a5123e753ff55a151e6f96db436e` 1339 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-node-linux-ppc64le.tar.gz) | `2e5bdf3a64b5387fe60aff1b05ccf0f822a3955815576f979e2e8b7673373c49b602ab53fc1a63dd4467787e9b62f3f369f4a656812078709be795e85aaf253b` 1340 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-node-linux-s390x.tar.gz) | `44632772ac2e13b91e9b04c05c42522f8fdcf574daff6e98719724ed0e07c44e1d01c0f047bf462e00f092a21b617822063ee6cc987c531fc7f0980c6b1319ba` 1341 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.3/kubernetes-node-windows-amd64.tar.gz) | `eb6128b0aa1001b0fea5d458692c96191bfaa51c4a8736a9c370e38fffb9d72a28937581cf79a1ea529058f039a08a3b8a23e7372c7f8a8265a5f9acd252a4c1` 1342 1343 ## Changelog since v1.16.2 1344 1345 ### Other notable changes 1346 1347 * kubeadm: fix skipped etcd upgrade on secondary control-plane nodes when the command "kubeadm upgrade node" is used. ([#85024](https://github.com/kubernetes/kubernetes/pull/85024), [@neolit123](https://github.com/neolit123)) 1348 * Fix kubelet metrics gathering on non-English Windows hosts ([#84156](https://github.com/kubernetes/kubernetes/pull/84156), [@wawa0210](https://github.com/wawa0210)) 1349 * Bump metrics-server to v0.3.5 ([#83015](https://github.com/kubernetes/kubernetes/pull/83015), [@olagacek](https://github.com/olagacek)) 1350 * Bumps metrics-server version to v0.3.6 with following bugfix: ([#83907](https://github.com/kubernetes/kubernetes/pull/83907), [@olagacek](https://github.com/olagacek)) 1351 * Don't break metric storage when duplicate pod metrics encountered causing hpa to fail 1352 * kube-apiserver: Fixed a regression accepting patch requests > 1MB ([#84963](https://github.com/kubernetes/kubernetes/pull/84963), [@liggitt](https://github.com/liggitt)) 1353 * kube-apiserver: fixed a bug that could cause a goroutine leak if the apiserver encountered an encoding error serving a watch to a websocket watcher ([#84693](https://github.com/kubernetes/kubernetes/pull/84693), [@tedyu](https://github.com/tedyu)) 1354 * azure: Add allow unsafe read from cache ([#83685](https://github.com/kubernetes/kubernetes/pull/83685), [@aramase](https://github.com/aramase)) 1355 * Add data cache flushing during unmount device for GCE-PD driver in Windows Server. ([#83591](https://github.com/kubernetes/kubernetes/pull/83591), [@jingxu97](https://github.com/jingxu97)) 1356 * Fixed binding of block PersistentVolumes / PersistentVolumeClaims when BlockVolume feature is off. ([#84049](https://github.com/kubernetes/kubernetes/pull/84049), [@jsafrane](https://github.com/jsafrane)) 1357 * kube-scheduler now fallbacks to emitting events using core/v1 Events when events.k8s.io/v1beta1 is disabled. ([#83692](https://github.com/kubernetes/kubernetes/pull/83692), [@yastij](https://github.com/yastij)) 1358 * Adds a metric apiserver_request_error_total to kube-apiserver. This metric tallies the number of request_errors encountered by verb, group, version, resource, subresource, scope, component, and code. ([#83427](https://github.com/kubernetes/kubernetes/pull/83427), [@logicalhan](https://github.com/logicalhan)) 1359 * Update Cluster Autoscaler version to 1.16.2 (CA release docs: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.16.2) ([#84038](https://github.com/kubernetes/kubernetes/pull/84038), [@losipiuk](https://github.com/losipiuk)) 1360 * Fixed an issue with informers missing an `Added` event if a recently deleted object was immediately recreated at the same time the informer dropped a watch and relisted. ([#83911](https://github.com/kubernetes/kubernetes/pull/83911), [@matte21](https://github.com/matte21)) 1361 * CSI detach timeout increased from 10 seconds to 2 minutes ([#84321](https://github.com/kubernetes/kubernetes/pull/84321), [@cduchesne](https://github.com/cduchesne)) 1362 * Switched intstr.Type to sized integer to follow API guidelines and improve compatibility with proto libraries ([#83956](https://github.com/kubernetes/kubernetes/pull/83956), [@liggitt](https://github.com/liggitt)) 1363 * Upgrade to etcd client 3.3.17 to fix bug where etcd client does not parse IPv6 addresses correctly when members are joining, and to fix bug where failover on multi-member etcd cluster fails certificate check on DNS mismatch ([#83968](https://github.com/kubernetes/kubernetes/pull/83968), [@jpbetz](https://github.com/jpbetz)) 1364 * Update to use go1.12.12 ([#84064](https://github.com/kubernetes/kubernetes/pull/84064), [@cblecker](https://github.com/cblecker)) 1365 * Fix handling tombstones in pod-disruption-budged controller. ([#83951](https://github.com/kubernetes/kubernetes/pull/83951), [@zouyee](https://github.com/zouyee)) 1366 1367 1368 1369 # v1.16.2 1370 1371 [Documentation](https://docs.k8s.io) 1372 1373 ## Downloads for v1.16.2 1374 1375 1376 filename | sha512 hash 1377 -------- | ----------- 1378 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes.tar.gz) | `9e21ab6317fe209dabac72444273b493f5f7639f73c489cc233da3a0205a93f8779964fe2cf2237bb6ca7535560c7c8ac48e25e75a1fbe97dc5b51f248ed7eeb` 1379 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-src.tar.gz) | `2e52029a8669f51020f689b0da481f56ab922ecd2c4495e37cb8456e9b10fee50f97bf9faa6a3bc947e4142aa4e5374d7f905a22e5f18830610eacfdf47adbc6` 1380 1381 ### Client Binaries 1382 1383 filename | sha512 hash 1384 -------- | ----------- 1385 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-darwin-386.tar.gz) | `6424b3047c5d7557b6c5536f29139bb6f71e4357b5e181d62bfd5fbf7d5575c9d5eeeab531b0afcc72ac17dd6609162bebc1045932ee58cbeb3093a4107cbf3a` 1386 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-darwin-amd64.tar.gz) | `791d2a4f965130ea1d7033f807b29015520afa317f97643f27480a6530213fd3fa06817ba9ce04e81f843b8420f05baa25e597e5880f99cc2ffbfc8b8ebf18cf` 1387 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-linux-386.tar.gz) | `fd727fe3200975f71d1f5b21991e5b8d2471e92dc3185df29b8ef21a9aa87d270baa1bd2f9f2dbe61abe9e9541ee806709d6ed08aa13254a5953ddc1f1ddb5a6` 1388 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-linux-amd64.tar.gz) | `69bb92c9b16c0286d7401d87cc73b85c88d6f9a17d2cf1748060e44525194b5be860daf7554c4c6319a546c9ff10f2b4df42a27e32d8f95a4052993b17ef57c0` 1389 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-linux-arm.tar.gz) | `f846f6018eb3d82741e6d2e8fd2f9f7ca8768ead56c3e2c330ba7e7dbba3209dd66f793b37d330134d3e8f38a03d6492e0054e55fcf32f74743b7b26f3159210` 1390 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-linux-arm64.tar.gz) | `49cb3495fa8bbe64cefdb9aab83d4cfbe6a0f3a0e58b1685a49244df02e5ee8bbfaf0be133f5cf5231b1470bb62bd699cd6aba8072cb6ea3542f3ccfd42d3813` 1391 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-linux-ppc64le.tar.gz) | `35a8b25309150cf5e1dab2af1d5118a24d296e21cdaf4bdec615badbc27a109ffb8b0e1cd93439fb728ec4905178eae2eb9d2ec2760053211b495c103d76fe3b` 1392 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-linux-s390x.tar.gz) | `ffd5ab012d38b7a7b9e0acd8339c30545a60bcabba5642f5020464846391ae41930e84c0103231f850f0f0c86e064c18f47bff95ccf11b410c0216a4d887b2fb` 1393 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-windows-386.tar.gz) | `c04d33b31734c7b25c54daa361986fcb745178320daa54d76b7627c475346a87c65443f7b139e1c86436b27eb9d10ed4a61ba5a64e0420bb6ce2f68fbad0a1b0` 1394 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-client-windows-amd64.tar.gz) | `e81fcf83523dd1dc79de7b498df6115465eb83d31a400853cdf143870a281322f24be5609d0c541810f90047a3c4e75530b9a6e872e51ed00d268530f2ed7ca9` 1395 1396 ### Server Binaries 1397 1398 filename | sha512 hash 1399 -------- | ----------- 1400 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-server-linux-amd64.tar.gz) | `4aa3da6d146db878f4605ea6334925e925268808856f482ea7f9d131dce76ee30d4ba077a6b35de34a68d8d9ca005cccd2e7e4866b775c1f7eaba1320a1b8ec5` 1401 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-server-linux-arm.tar.gz) | `0a9cea07a1172038d300277f1d305d99b147ae5949df3dd63856944cc0e4724105a0d79bf9ea39dface923d794b71e9f40d4104218167e56096a9e82a422864a` 1402 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-server-linux-arm64.tar.gz) | `b1aadb87ff310aec7292df5386613c4c0036c0e3c20e0a4ab7bd3d5eea8fff0116c8b8b47289e894950b249be4207daac58449ec9a587353873890b47217ecd8` 1403 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-server-linux-ppc64le.tar.gz) | `2aecda7fb53efd23c34cfe90ae37d55efec513996573c9124d9bf94809ea5a86bcd786e84d646039fbaff2e42d7e8f9c5db179beec3fac1f0a420920c93dce35` 1404 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-server-linux-s390x.tar.gz) | `74fd3d5dbedac211570ef011f14e98d37848587b670038c69354c0162668aa2c6789835a5dbfc82f43fb9662444756f974de75917f75f59ab51e6ae9183e5c20` 1405 1406 ### Node Binaries 1407 1408 filename | sha512 hash 1409 -------- | ----------- 1410 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-node-linux-amd64.tar.gz) | `d1e139006e1ce6b66a1c0b9dd90509db712e5e024e1c5a4c0a1bb185211dd138cbcf201e6a9cdf7df9b78b6817523da23097c8396c0afb98481052447d329cc9` 1411 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-node-linux-arm.tar.gz) | `5779c23d4f77edf5d5c3b8bb55fd190bf180f13c782e44a50a96572477af30008685c4e670c07ceb56c8e64390be7e7cbf14a9b25c894c66f90131a993bda4fc` 1412 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-node-linux-arm64.tar.gz) | `f09bc220a05b8f5fca6d4575198ac2b7625fcbceb09ffd6da587c8b13a5b93365b601b94a25fb8cc360c7521ccc30d11d86d88490f5a48369f6078c4395593c2` 1413 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-node-linux-ppc64le.tar.gz) | `96b3f8578e4f5e5261489ba392e7167bfdbe5ec6fb60a369455edffbc027b3086bd99055f7e3cabbf56426b7b181764d5cd5979bacd3ad8c26b541cd94ce62d8` 1414 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-node-linux-s390x.tar.gz) | `4c9c7e541164781231378091cae0ab693c29b517b7737be191436c5e85ea4702cdd640890ae7c1295273270cb7944e2a250e2791a9a07bb415dac67d3129d1c1` 1415 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.2/kubernetes-node-windows-amd64.tar.gz) | `a88e7a1c6f72ea6073dbb4ddfe2e7c8bd37c9a56d94a33823f531e303a9915e7a844ac5880097724e44dfa7f4a9659d14b79cc46e2067f6b13e6df3f3f1b0f64` 1416 1417 ## Changelog since v1.16.1 1418 1419 ### Other notable changes 1420 1421 * Fixed panic when accessing CustomResources of a CRD with x-kubernetes-int-or-string. ([#83789](https://github.com/kubernetes/kubernetes/pull/83789), [@sttts](https://github.com/sttts)) 1422 * Fixed a bug in the single-numa-node policy of the TopologyManager. Previously, pods that only requested CPU resources and did not request any third-party devices would fail to launch with a TopologyAffinity error. Now they will launch successfully. ([#83697](https://github.com/kubernetes/kubernetes/pull/83697), [@klueska](https://github.com/klueska)) 1423 * Fixes kube-proxy bug accessing self nodeip:port on windows ([#83027](https://github.com/kubernetes/kubernetes/pull/83027), [@liggitt](https://github.com/liggitt)) 1424 * Fix error where metrics related to dynamic kubelet config isn't registered ([#83184](https://github.com/kubernetes/kubernetes/pull/83184), [@odinuge](https://github.com/odinuge)) 1425 * Use online nodes instead of possible nodes when discovering available NUMA nodes ([#83196](https://github.com/kubernetes/kubernetes/pull/83196), [@zouyee](https://github.com/zouyee)) 1426 * Update Azure load balancer to prevent orphaned public IP addresses ([#82890](https://github.com/kubernetes/kubernetes/pull/82890), [@chewong](https://github.com/chewong)) 1427 * Fixes a goroutine leak in kube-apiserver when a request times out. ([#83333](https://github.com/kubernetes/kubernetes/pull/83333), [@lavalamp](https://github.com/lavalamp)) 1428 * Fix aggressive VM calls for Azure VMSS ([#83102](https://github.com/kubernetes/kubernetes/pull/83102), [@feiskyer](https://github.com/feiskyer)) 1429 * Fixes a flaw (CVE-2019-11253) in json/yaml decoding where large or malformed documents could consume excessive server resources. Request bodies for normal API requests (create/delete/update/patch operations of regular resources) are now limited to 3MB. ([#83261](https://github.com/kubernetes/kubernetes/pull/83261), [@liggitt](https://github.com/liggitt)) 1430 1431 1432 1433 # v1.16.1 1434 1435 [Documentation](https://docs.k8s.io) 1436 1437 ## Downloads for v1.16.1 1438 1439 1440 filename | sha512 hash 1441 -------- | ----------- 1442 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes.tar.gz) | `3dc4f0f2a208d3f235b4dc579418c503ba64fec261294c3683268259117e5c82f56db3f0c4ad63199f0f35f0de21df0b13ca00a74efe7b3142b275f0733e9219` 1443 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-src.tar.gz) | `2d79b1e4e43c57c0e24da07f1312d0e49f8c43ebba0231dd5d628a52c25e929fed34c3d34b2ab739c0348c2f42c88feea8f1a4c2d821ace170f451bf74903ae3` 1444 1445 ### Client Binaries 1446 1447 filename | sha512 hash 1448 -------- | ----------- 1449 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-darwin-386.tar.gz) | `f7139df5f06f3fc9d1944058e7ba40a482621d6a169bc2f09a50dfa8f903866706051cded14fbd39d8d03c576f9e5219b0887f029ee1cac45919dc5ff232cdcd` 1450 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-darwin-amd64.tar.gz) | `c4d458a0c3ef2dc9e4527daecf6b34da065d1ae7fd245f8b8aceca43467f03073f538e65e504f9800d3db0c64d88fc41f687b493be4d9a8283f39dd001a86a7d` 1451 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-linux-386.tar.gz) | `82d79bc833ae5317c4a1d0d397ccaa9aef616e0c2e5ff2166da8c66a4f55851f4bfa3b57708f75dc12eabefdee35b08d8f7bee74cfaf38a0339ee927f90e1655` 1452 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-linux-amd64.tar.gz) | `e355a74a17d96785b0b217673e67fa0f02daa1939f10d410602ac0a0d061a4db71d727b67f75aa886007dab95dd5c7f8cc38253d291dc4d2504ce673df69fb32` 1453 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-linux-arm.tar.gz) | `b9fba5995a7a02d001bead0305ae2a0c8ee7d2a98069aad6e57612988e298972cb6f69267afb79df292195958871c56a2cc5ec7a8e180a8d56c428f82bbbece8` 1454 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-linux-arm64.tar.gz) | `64917bd6c4b277f4cb676b2e3d69cb12148bc90e2b04011d11985e492d525646fd13021b74f93914d37849fba87c07c513fafc1bd07edc7d3e4415fe8bd9ea37` 1455 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-linux-ppc64le.tar.gz) | `258cc1b13f3ef90c52d401190f4d59c8c0d10367a6d960c6273522a15b879de2df4d141461d80338144e83995422da802a417a55911d7a4d8aeeec5aecd2adec` 1456 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-linux-s390x.tar.gz) | `adaa34bde363a030f099a96417e257688e6280d86bdd3e04f7c06b6bccce639f599d494b5181a78e91c7f50207c53fa1a04b3b2fbd65be91512e51441e66a2e8` 1457 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-windows-386.tar.gz) | `3ed99240a99a2d9f4193264f14168896a2418f296e5a2372e4e6aac44b65cd6a531ceec71d24333a7c825ba02acec63fdb51b97aa9fb4665048e929b7e4b774d` 1458 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-client-windows-amd64.tar.gz) | `cbf8363147e44bfadee4da9ced9ab7bfc3c88ba7dcfa3f64803376efa5adc6dbb38519f73253aa9ea8b12bb42a92f5ea78e8b63fb082e4ace04d3ee3e285f549` 1459 1460 ### Server Binaries 1461 1462 filename | sha512 hash 1463 -------- | ----------- 1464 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-server-linux-amd64.tar.gz) | `c43795fc2bfaee240f1e32e0b523cbbc8a888a40efa85371e682433d6f60dd1f3225ca34484fc9db54e68e250acbb5c2c388071bcd033fed0d8a47d044708b0b` 1465 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-server-linux-arm.tar.gz) | `3a6c38db6d6c0950f6d3cd4c158ba9c816dcbf1b1959a74d55ead53b1df2cf9131f56d5b0f5720f9c2000085ada1fd4d48439ea20f4ba35764e1a54f94848697` 1466 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-server-linux-arm64.tar.gz) | `3cbde62bb5ef0af033d643cea800e891c199f9d6e9f0a3e1131bebe6759fd72b2c8d1f86d92ab5d1c352297b57320b47c70a6422768656b2c44c4ef4b7e3c721` 1467 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-server-linux-ppc64le.tar.gz) | `a60a25a346de61e705d416b50f8eac582a36faa622c6c45d4e3af92c46f48409b4c2a9bcbfd9297cc77d747cf1a96721146a24a48500a55b806c6fc0d9008e21` 1468 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-server-linux-s390x.tar.gz) | `1ee9bbb36245825604d7cf4c9bdb2e43d0d755958413809ae7d3c9448ada567c0ab510f68318055dd71ec73d2ccce5b8d23faa698fd42772e3bd3c9592006d80` 1469 1470 ### Node Binaries 1471 1472 filename | sha512 hash 1473 -------- | ----------- 1474 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-node-linux-amd64.tar.gz) | `4ef686de6aa5a2bdcb8034b3def008995fc4474f34f939620c657ae115c61403c8d5698c9af68d3fe7015beaa8d578edc9f1a62b08dea92d99a3c73c9fe9e7a6` 1475 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-node-linux-arm.tar.gz) | `58607b1cf68e8420bacc8dc3f95b19b7f90fe4fda1255814c76e66f67638b8db150eb464f85e1b625e13a74b0d86a973e54da87e2fb41985eff14b25a7761b1d` 1476 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-node-linux-arm64.tar.gz) | `f02417299b5b6f3ee4b325d4a23b0ca19ee3c10dbe871a716470e68a3e89f4b84232eac1048dc9dc88a00ed069a8fd3b4f7817664e296e8842322379702e67e4` 1477 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-node-linux-ppc64le.tar.gz) | `fa3b44b668d0a341847601a18c463698810457e113459d6f8ddbce6f9305b613c4f61f9831e16d8bab128efdd69524c86981785f9703e248eff11f95e9330e74` 1478 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-node-linux-s390x.tar.gz) | `afe166d9245bd35fbc21763b9ef24bf174396c0611f75ddf93f1e4f3fb3e1cf1d4e7998d963e26fe3761bf4984ea33a5d6656035092c9b0d35e94f97882a5595` 1479 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.1/kubernetes-node-windows-amd64.tar.gz) | `93ebe21c147dcb38cb1ee222975f35f7c8c8175255cfce09381c8852734d054def58bc630cf868d99baa7f90bd2c201ccaa42dbae3ef360c3135c825ec2c74b1` 1480 1481 ## Changelog since v1.16.0 1482 1483 ### Other notable changes 1484 1485 * Limit the body length of exec readiness/liveness probes. remote CRIs and Docker shim read a max of 16MB output of which the exec probe itself inspects 10kb. ([#82514](https://github.com/kubernetes/kubernetes/pull/82514), [@dims](https://github.com/dims)) 1486 * Fix possible fd leak and closing of dirs when using openstack ([#82873](https://github.com/kubernetes/kubernetes/pull/82873), [@odinuge](https://github.com/odinuge)) 1487 * Update to go 1.12.10 ([#83139](https://github.com/kubernetes/kubernetes/pull/83139), [@cblecker](https://github.com/cblecker)) 1488 * Use ipv4 in wincat port forward. ([#83036](https://github.com/kubernetes/kubernetes/pull/83036), [@liyanhui1228](https://github.com/liyanhui1228)) 1489 * Fixes a panic in kube-controller-manager cleaning up bootstrap tokens ([#82887](https://github.com/kubernetes/kubernetes/pull/82887), [@tedyu](https://github.com/tedyu)) 1490 * Resolves bottleneck in internal API server communication that can cause increased goroutines and degrade API Server performance ([#80465](https://github.com/kubernetes/kubernetes/pull/80465), [@answer1991](https://github.com/answer1991)) 1491 * Resolves regression generating informers for packages whose names contain `.` characters ([#82410](https://github.com/kubernetes/kubernetes/pull/82410), [@nikhita](https://github.com/nikhita)) 1492 * Fixed a scheduler panic when using PodAffinity. ([#82841](https://github.com/kubernetes/kubernetes/pull/82841), [@Huang-Wei](https://github.com/Huang-Wei)) 1493 * Update Cluster Autoscaler version to 1.16.1 (release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.16.1) ([#83052](https://github.com/kubernetes/kubernetes/pull/83052), [@losipiuk](https://github.com/losipiuk)) 1494 * Resolves issue with /readyz and /livez not including etcd and kms health checks ([#82713](https://github.com/kubernetes/kubernetes/pull/82713), [@logicalhan](https://github.com/logicalhan)) 1495 * fix: azure disk detach failure if node not exists ([#82640](https://github.com/kubernetes/kubernetes/pull/82640), [@andyzhangx](https://github.com/andyzhangx)) 1496 1497 1498 1499 # v1.16.0 1500 1501 [Documentation](https://docs.k8s.io) 1502 1503 ## Downloads for v1.16.0 1504 1505 1506 filename | sha512 hash 1507 -------- | ----------- 1508 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes.tar.gz) | `99aa74225dd999d112ebc3e7b7d586a2312ec9c99de7a7fef8bbbfb198a5b4cf740baa57ea262995303e2a5060d26397775d928a086acd926042a41ef00f200b` 1509 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-src.tar.gz) | `0be7d1d6564385cc20ff4d26bab55b71cc8657cf795429d04caa5db133a6725108d6a116553bf55081ccd854a4078e84d26366022634cdbfffd1a34a10b566cf` 1510 1511 ### Client Binaries 1512 1513 filename | sha512 hash 1514 -------- | ----------- 1515 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-darwin-386.tar.gz) | `a5fb80d26c2a75741ad0efccdacd5d5869fbc303ae4bb1920a6883ebd93a6b41969f898d177f2602faf23a7462867e1235edeb0ba0675041d0c8d5ab266ec62d` 1516 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-darwin-amd64.tar.gz) | `47a9a78fada4b840d9ae4dac2b469a36d0812ac83d22fd798c4cb0f1673fb65c6558383c19a7268ed7101ac9fa32d53d79498407bdf94923f4f8f019ea39e912` 1517 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-linux-386.tar.gz) | `916e4dd98f5ed8ee111eeb6c2cf5c5f313e1d98f3531b40a5a777240ddb96b9cc53df101daa077ffff52cf01167fdcc39d38a8655631bac846641308634e127a` 1518 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-linux-amd64.tar.gz) | `fccf152588edbaaa21ca94c67408b8754f8bc55e49470380e10cf987be27495a8411d019d807df2b2c1c7620f8535e8f237848c3c1ac3791b91da8df59dea5aa` 1519 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-linux-arm.tar.gz) | `066c55fabbe3434604c46574c51c324336a02a5bfaed2e4d83b67012d26bf98354928c9c12758b53ece16b8567e2b5ce6cb88d5cf3008c7baf3c5df02611a610` 1520 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-linux-arm64.tar.gz) | `e41be74cc36240a64ecc962a066988b5ef7c3f3112977efd4e307b35dd78688f41d6c5b376a6d1152d843182bbbe75d179de75675548bb846f8c1e28827e0e0c` 1521 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-linux-ppc64le.tar.gz) | `08783eb3bb2e35b48dab3481e17d6e345d43bab8b8dee25bb5ff184ba46cb632750d4c38e9982366050aecce6e121c67bb6812dbfd607216acd3a2d19e05f5a1` 1522 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-linux-s390x.tar.gz) | `bcb6eb9cd3d8c92dfaf4f102ff2dc7517f632b1e955be6a02e7f223b15fc09c4ca2d6d9cd5b23871168cf6b455e2368daf17025c9cd61bf43d2ea72676db913a` 1523 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-windows-386.tar.gz) | `efbc764d8e2889ce13c9eaaa61f685a8714563ddc20464523140d6f5bef0dfd51b745c3bd3fe2093258db242bf9b3207f8e9f451b0484de64f18cdb7162ec30e` 1524 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-client-windows-amd64.tar.gz) | `b34bce694c6a0e4c8c5ddabcecb6adcb4d35f8c126b4b5ced7e44ef39cd45982dd9f6483a38e04430846f4da592dc74b475c37da7fe08444ef4eb5efde85e0b2` 1525 1526 ### Server Binaries 1527 1528 filename | sha512 hash 1529 -------- | ----------- 1530 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-server-linux-amd64.tar.gz) | `a6bdac1eba1b87dc98b2bf5bf3690758960ecb50ed067736459b757fca0c3b01dd01fd215b4c06a653964048c6a81ea80b61ee8c7e4c98241409c091faf0cee1` 1531 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-server-linux-arm.tar.gz) | `0560e1e893fe175d74465065d43081ee7f40ba7e7d7cafa53e5d7491f89c61957cf0d3abfa4620cd0f33b6e44911b43184199761005d20b72e3cd2ddc1224f9f` 1532 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-server-linux-arm64.tar.gz) | `4d5dd001fa3ac2b28bfee64e85dbedab0706302ffd634c34330617674e7a90e0108710f4248a2145676bd72f0bbc3598ed61e1e739c64147ea00d3b6a4ba4604` 1533 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-server-linux-ppc64le.tar.gz) | `cc642fca57e22bf6edd371e61e254b369b760c67fa00cac50e34464470f7eea624953deff800fa1e4f7791fe06791c48dbba3ed47e789297ead889c2aa7b2bbf` 1534 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-server-linux-s390x.tar.gz) | `1f480ba6f593a3aa20203e82e9e34ac206e35839fd9135f495c5d154480c57d1118673dcb5a6b112c18025fb4a847f65dc7aac470f01d2f06ad3da6aa63d98a3` 1535 1536 ### Node Binaries 1537 1538 filename | sha512 hash 1539 -------- | ----------- 1540 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-node-linux-amd64.tar.gz) | `e987f141bc0a248e99a371ce220403b78678c739a39dad1c1612e63a0bee4525fbca5ee8c2b5e5332a553cc5f63bce9ec95645589298f41fe83e1fd41faa538e` 1541 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-node-linux-arm.tar.gz) | `8b084c1063beda2dd4000e8004634d82e580f05cc300c2ee13ad84bb884987b2c7fd1f033fb2ed46941dfc311249acef06efe5044fb72dc4b6089c66388e1f61` 1542 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-node-linux-arm64.tar.gz) | `365bdf9759e24d22cf507a0a5a507895ed44723496985e6d8f0bd10b03ffe7c78198732ee39873912147f2dd840d2e284118fc6fc1e3876d8f4c2c3a441def0b` 1543 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-node-linux-ppc64le.tar.gz) | `ff54d83dd0fd3c447cdd76cdffd253598f6800045d2b6b91b513849d15b0b602590002e7fe2a55dc25ed5a05787f4973c480126491d24be7c5fce6ce98d0b6b6` 1544 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-node-linux-s390x.tar.gz) | `527cd9bf4bf392c3f097f232264c0f0e096ac410b5211b0f308c9d964f86900f5875012353b0b787efc9104f51ad90880f118efb1da54eba5c7675c1840eae5f` 1545 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0/kubernetes-node-windows-amd64.tar.gz) | `4f76a94c70481dd1d57941f156f395df008835b5d1cc17708945e8f560234dbd426f3cff7586f10fd4c24e14e3dfdce28e90c8ec213c23d6ed726aec94e9b0ff` 1546 1547 # Kubernetes v1.16.0 Release Notes 1548 1549 A complete changelog for the release notes is now hosted in a customizable format at [relnotes.k8s.io](https://relnotes.k8s.io/?releaseVersions=1.16.0). Check it out and please give us your feedback! 1550 1551 ## What’s New (Major Themes) 1552 1553 We’re pleased to announce the delivery of Kubernetes 1.16, our third release of 2019! Kubernetes 1.16 consists of 31 enhancements: 8 enhancements moving to stable, 8 enhancements in beta, and 15 enhancements in alpha. 1554 1555 The main themes of this release are: 1556 1557 - **Custom resources:** CRDs are in widespread use as a way to extend Kubernetes to persist and serve new resource types, and have been available in beta since the 1.7 release. The 1.16 release marks the graduation of CRDs to general availability (GA). 1558 - **Admission webhooks:** Admission webhooks are in widespread use as a Kubernetes extensibility mechanism and have been available in beta since the 1.9 release. The 1.16 release marks the graduation of admission webhooks to general availability (GA). 1559 - **Overhauled metrics**: Kubernetes has previously made extensive use of a global metrics registry to register metrics to be exposed. By implementing a metrics registry, metrics are registered in more transparent means. Previously, Kubernetes metrics have been excluded from any kind of stability requirements. 1560 - **Volume Extension**: There are quite a few enhancements in this release that pertain to volumes and volume modifications. Volume resizing support in CSI specs is moving to beta which allows for any CSI spec volume plugin to be resizable. 1561 1562 ### Additional Notable Feature Updates 1563 1564 - [Topology Manager](https://github.com/kubernetes/enhancements/issues/693), a new Kubelet component, aims to co-ordinate resource assignment decisions to provide optimized resource allocations. 1565 - [IPv4/IPv6 dual-stack](https://kubernetes.io/docs/concepts/services-networking/dual-stack) enables the allocation of both IPv4 and IPv6 addresses to Pods and Services. 1566 - [API Server Network Proxy](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/1281-network-proxy/README.md) going alpha in 1.16. 1567 - [Extensions](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cloud-provider/991-cloud-controller-migration/README.md) for Cloud Controller Manager Migration. 1568 - Continued deprecation of extensions/v1beta1, apps/v1beta1, and apps/v1beta2 APIs; these extensions will be retired in 1.16! 1569 1570 ## Known Issues 1571 1572 - The etcd and KMS plugin health checks are not exposed in the new `livez` and `readyz` endpoints. This will be fixed in 1.16.1. 1573 - Systems running `iptables` 1.8.0 or newer should start it in legacy mode. Please note that this affects all versions of Kubernetes and not only v1.16.0. For more detailed information about the issue and how to apply a workaround, please refer to the official documentation 1574 - Generating informers for packages in directories containing dots in their name is broken. This will be fixed in v1.16.1. ([#82860](https://github.com/kubernetes/kubernetes/issues/82860)) 1575 - kube-scheduler won't be able to report scheduling Events if `events.k8s.io/v1beta1` API is disabled. We are targeting the fix for v1.16.2 ([#83203](https://github.com/kubernetes/kubernetes/issues/83203)) 1576 - The etcd client library vendored in v1.16 (github.com/coreos/etcd/clientv3) has a bug where IPv6 IP addresses get incorrectly trimmed to the first occurance of the `:` character. This affects all users of the client library including the kube-apiserver and kubeadm. We are targeting the fix for this bug in v1.16.3. ([#83550](https://github.com/kubernetes/kubernetes/issues/83550)) 1577 1578 ## Urgent Upgrade Notes 1579 1580 ### (No, really, you MUST read this before you upgrade) 1581 1582 #### Cluster Lifecycle 1583 1584 - Container images tar files for `amd64` will now contain the architecture in the RepoTags manifest.json section. 1585 If you are using docker manifests there are not visible changes. ([#80266](https://github.com/kubernetes/kubernetes/pull/80266), [@javier-b-perez](https://github.com/javier-b-perez)) 1586 - kubeadm now deletes the bootstrap-kubelet.conf file after TLS bootstrap 1587 User relying on bootstrap-kubelet.conf should switch to kubelet.conf that contains node credentials ([#80676](https://github.com/kubernetes/kubernetes/pull/80676), [@fabriziopandini](https://github.com/fabriziopandini)) 1588 - Node labels `beta.kubernetes.io/metadata-proxy-ready`, `beta.kubernetes.io/metadata-proxy-ready` and `beta.kubernetes.io/kube-proxy-ds-ready` are no longer added on new nodes. 1589 - ip-mask-agent addon starts to use the label `node.kubernetes.io/masq-agent-ds-ready` instead of `beta.kubernetes.io/masq-agent-ds-ready` as its node selector. 1590 - kube-proxy addon starts to use the label `node.kubernetes.io/kube-proxy-ds-ready` instead of `beta.kubernetes.io/kube-proxy-ds-ready` as its node selector. 1591 - metadata-proxy addon starts to use the label `cloud.google.com/metadata-proxy-ready` instead of `beta.kubernetes.io/metadata-proxy-ready` as its node selector. 1592 1593 #### Storage 1594 1595 - When PodInfoOnMount is enabled for a CSI driver, the new csi.storage.k8s.io/ephemeral parameter in the volume context allows a driver's NodePublishVolume implementation to determine on a case-by-case basis whether the volume is ephemeral or a normal persistent volume ([#79983](https://github.com/kubernetes/kubernetes/pull/79983), [@pohly](https://github.com/pohly)) 1596 - Add CSI Migration Shim for VerifyVolumesAreAttached and BulkVolumeVerify ([#80443](https://github.com/kubernetes/kubernetes/pull/80443), [@davidz627](https://github.com/davidz627)) 1597 - Promotes VolumePVCDataSource (Cloning) feature to beta for 1.16 release ([#81792](https://github.com/kubernetes/kubernetes/pull/81792), [@j-griffith](https://github.com/j-griffith)) 1598 - Integrated volume limits for in-tree and CSI volumes into one scheduler predicate. ([#77595](https://github.com/kubernetes/kubernetes/pull/77595), [@bertinatto](https://github.com/bertinatto)) 1599 1600 ## Deprecations and Removals 1601 1602 - API 1603 1604 - The following APIs are no longer served by default: 1605 1606 - All resources under `apps/v1beta1` and `apps/v1beta2` - use `apps/v1` instead 1607 - `daemonsets`, `deployments`, `replicasets` resources under `extensions/v1beta1` - use `apps/v1` instead 1608 - `networkpolicies` resources under `extensions/v1beta1` - use `networking.k8s.io/v1` instead 1609 - `podsecuritypolicies` resources under `extensions/v1beta1` - use `policy/v1beta1` instead 1610 1611 Serving these resources can be temporarily re-enabled using the `--runtime-config` apiserver flag. 1612 1613 - `apps/v1beta1=true` 1614 - `apps/v1beta2=true` 1615 - `extensions/v1beta1/daemonsets=true,extensions/v1beta1/deployments=true,extensions/v1beta1/replicasets=true,extensions/v1beta1/networkpolicies=true,extensions/v1beta1/podsecuritypolicies=true` 1616 1617 The ability to serve these resources will be completely removed in v1.18. ([#70672](https://github.com/kubernetes/kubernetes/pull/70672), [@liggitt](https://github.com/liggitt)) 1618 1619 - Ingress resources will no longer be served from `extensions/v1beta1` in v1.20. Migrate use to the `networking.k8s.io/v1beta1` API, available since v1.14. Existing persisted data can be retrieved via the `networking.k8s.io/v1beta1` API. 1620 - PriorityClass resources will no longer be served from `scheduling.k8s.io/v1beta1` and `scheduling.k8s.io/v1alpha1` in v1.17. Migrate to the `scheduling.k8s.io/v1` API, available since v1.14. Existing persisted data can be retrieved via the `scheduling.k8s.io/v1` API. 1621 - The `export` query parameter for list API calls, deprecated since v1.14, will be removed in v1.18. 1622 - The `series.state` field in the events.k8s.io/v1beta1 Event API is deprecated and will be removed in v1.18 ([#75987](https://github.com/kubernetes/kubernetes/pull/75987), [@yastij](https://github.com/yastij)) 1623 - The `apiextensions.k8s.io/v1beta1` version of `CustomResourceDefinition` is deprecated and will no longer be served in v1.19. Use `apiextensions.k8s.io/v1` instead. ([#79604](https://github.com/kubernetes/kubernetes/pull/79604), [@liggitt](https://github.com/liggitt)) 1624 - The `admissionregistration.k8s.io/v1beta1` versions of `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration` are deprecated and will no longer be served in v1.19. Use `admissionregistration.k8s.io/v1` instead. ([#79549](https://github.com/kubernetes/kubernetes/pull/79549), [@liggitt](https://github.com/liggitt)) 1625 - The alpha `metadata.initializers` field, deprecated in 1.13, has been removed. ([#79504](https://github.com/kubernetes/kubernetes/pull/79504), [@yue9944882](https://github.com/yue9944882)) 1626 - The deprecated node condition type `OutOfDisk` has been removed. Use the `DiskPressure` condition instead. ([#72420](https://github.com/kubernetes/kubernetes/pull/72420), [@Pingan2017](https://github.com/Pingan2017)) 1627 - The `metadata.selfLink` field is deprecated in individual and list objects. It will no longer be returned starting in v1.20, and the field will be removed entirely in v1.21. ([#80978](https://github.com/kubernetes/kubernetes/pull/80978), [@wojtek-t](https://github.com/wojtek-t)) 1628 - The deprecated cloud providers `ovirt`, `cloudstack` and `photon` have been removed ([#72178](https://github.com/kubernetes/kubernetes/pull/72178), [@dims](https://github.com/dims)) 1629 - The `Cinder` and `ScaleIO` volume providers have been deprecated and will be removed in a future release. ([#80099](https://github.com/kubernetes/kubernetes/pull/80099), [@dims](https://github.com/dims)) 1630 - The GA `PodPriority` feature gate is now on by default and cannot be disabled. The feature gate will be removed in v1.18. ([#79262](https://github.com/kubernetes/kubernetes/pull/79262), [@draveness](https://github.com/draveness)) 1631 - Aggregated discovery requests can now timeout. Aggregated API servers must complete discovery calls within 5 seconds (other requests can take longer). Use the feature gate `EnableAggregatedDiscoveryTimeout=false` to temporarily revert behavior to the previous 30 second timeout if required (the temporary `EnableAggregatedDiscoveryTimeout` feature gate will be removed in v1.17). ([#82146](https://github.com/kubernetes/kubernetes/pull/82146), [@deads2k](https://github.com/deads2k)) 1632 - the `scheduler.alpha.kubernetes.io/critical-pod` annotation is removed. Pod priority (`spec.priorityClassName`) should be used instead to mark pods as critical. ([#80342](https://github.com/kubernetes/kubernetes/pull/80342), [@draveness](https://github.com/draveness)) 1633 - the NormalizeScore plugin set is removed from scheduler framework config API. Use ScorePlugin only. ([#80930](https://github.com/kubernetes/kubernetes/pull/80930), [@liu-cong](https://github.com/liu-cong)) 1634 - the node label `alpha.service-controller.kubernetes.io/exclude-balancer` which excludes a node from cloud load balancers (using Service Type=LoadBalancer) is deprecated in favor of `node.kubernetes.io/exclude-balancer`. Support for `alpha.service-controller.kubernetes.io/exclude-balancer` will be removed in v1.18. 1635 1636 - Features: 1637 1638 - The following features are now GA, and the associated feature gates are deprecated and will be removed in v1.17: 1639 - `GCERegionalPersistentDisk` (since 1.15.0) 1640 - `CustomResourcePublishOpenAPI` 1641 - `CustomResourceSubresources` 1642 - `CustomResourceValidation` 1643 - `CustomResourceWebhookConversion` 1644 - The feature flags `HugePages`, `VolumeScheduling`, `CustomPodDNS` and `PodReadinessGates` have been removed ([#79307](https://github.com/kubernetes/kubernetes/pull/79307), [@draveness](https://github.com/draveness)) 1645 1646 - hyperkube 1647 1648 - the `--make-symlinks` flag, deprecated in v1.14, has been removed. ([#80017](https://github.com/kubernetes/kubernetes/pull/80017), [@Pothulapati](https://github.com/Pothulapati)) 1649 1650 - kube-apiserver 1651 1652 - the `--basic-auth-file` flag and authentication mode is deprecated and will be removed in a future release. It is not recommended for production environments. ([#81152](https://github.com/kubernetes/kubernetes/pull/81152), [@tedyu](https://github.com/tedyu)) 1653 - the `--cloud-provider-gce-lb-src-cidrs` flag has been deprecated. This flag will be removed once the GCE Cloud Provider is removed from kube-apiserver. ([#81094](https://github.com/kubernetes/kubernetes/pull/81094), [@andrewsykim](https://github.com/andrewsykim)) 1654 - the `--enable-logs-handler` flag and log-serving functionality is deprecated since v1.15, and scheduled to be removed in v1.19. ([#77611](https://github.com/kubernetes/kubernetes/pull/77611), [@rohitsardesai83](https://github.com/rohitsardesai83)) 1655 - Deprecate the default service IP CIDR. The previous default was `10.0.0.0/24` which will be removed in 6 months/2 releases. Cluster admins must specify their own desired value, by using `--service-cluster-ip-range` on kube-apiserver. ([#81668](https://github.com/kubernetes/kubernetes/pull/81668), [@darshanime](https://github.com/darshanime)) 1656 1657 - kube-proxy 1658 1659 - the `--resource-container` flag has been removed from kube-proxy, and specifying it will now cause an error. The behavior is now as if you specified `--resource-container=""`. If you previously specified a non-empty `--resource-container`, you can no longer do so as of kubernetes 1.16. ([#78294](https://github.com/kubernetes/kubernetes/pull/78294), [@vllry](https://github.com/vllry)) 1660 1661 - kube-scheduler 1662 1663 - Migrate scheduler to use v1beta1 Event API. any tool targeting scheduler events needs to use v1beta1 Event API ([#78447](https://github.com/kubernetes/kubernetes/pull/78447), [@yastij](https://github.com/yastij)) 1664 1665 - kubeadm 1666 1667 - The CoreDNS Deployment now checks readiness via the `ready` plugin. 1668 - The `proxy` plugin has been deprecated. The `forward` plugin is to be used instead. 1669 - `kubernetes` plugin removes the `resyncperiod` option. 1670 - The `upstream` option is deprecated and ignored if included. 1671 ([#82127](https://github.com/kubernetes/kubernetes/pull/82127), [@rajansandeep](https://github.com/rajansandeep)) 1672 1673 - kubectl 1674 1675 - `kubectl convert`, deprecated since v1.14, will be removed in v1.17. 1676 - The `--export` flag for the `kubectl get` command, deprecated since v1.14, will be removed in v1.18. 1677 - `kubectl cp` no longer supports copying symbolic links from containers; to support this use case, see `kubectl exec --help` for examples using `tar` directly ([#82143](https://github.com/kubernetes/kubernetes/pull/82143), [@soltysh](https://github.com/soltysh)) 1678 - Removed deprecated flag `--include-uninitialized`. ([#80337](https://github.com/kubernetes/kubernetes/pull/80337), [@draveness](https://github.com/draveness)) 1679 1680 - kubelet 1681 1682 - the `--containerized` flag was deprecated in 1.14 and has been removed ([#80043](https://github.com/kubernetes/kubernetes/pull/80043), [@dims](https://github.com/dims)) 1683 - the `beta.kubernetes.io/os` and `beta.kubernetes.io/arch` labels, deprecated since v1.14, are targeted for removal in v1.18. 1684 - cAdvisor json endpoints have been deprecated since 1.15. ([#78504](https://github.com/kubernetes/kubernetes/pull/78504), [@dashpole](https://github.com/dashpole)) 1685 - removed the ability to set `kubernetes.io`- or `k8s.io`-prefixed labels via `--node-labels`, other than the [specifically allowed labels/prefixes](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/279-limit-node-access/README.md#proposal). ([#79305](https://github.com/kubernetes/kubernetes/pull/79305), [@paivagustavo](https://github.com/paivagustavo)) 1686 1687 - client-go 1688 - Remove `DirectCodecFactory` (replaced with `serializer.WithoutConversionCodecFactory`), `DirectEncoder` (replaced with `runtime.WithVersionEncoder`) and `DirectDecoder` (replaced with `runtime.WithoutVersionDecoder`). ([#79263](https://github.com/kubernetes/kubernetes/pull/79263), [@draveness](https://github.com/draveness)) 1689 1690 ## Metrics Changes 1691 1692 ### Added metrics 1693 1694 - Added metrics `aggregator_openapi_v2_regeneration_count`, `aggregator_openapi_v2_regeneration_gauge` and `apiextension_openapi_v2_regeneration_count` counting the triggering APIService and CRDs and the reason (add, update, delete) when kube-apiserver regenerates the OpenAPI spec. ([#81786](https://github.com/kubernetes/kubernetes/pull/81786), [@sttts](https://github.com/sttts)) 1695 - Added metrics `authentication_attempts` that can be used to understand the attempts of authentication. ([#81509](https://github.com/kubernetes/kubernetes/pull/81509), [@RainbowMango](https://github.com/RainbowMango)) 1696 - Add a new counter metrics `apiserver_admission_webhook_rejection_count` with details about the causing for a webhook rejection. ([#81399](https://github.com/kubernetes/kubernetes/pull/81399), [@roycaihw](https://github.com/roycaihw)) 1697 - NFS Drivers are now enabled to collect metrics, StatFS metrics provider is used to collect the metrics. (@brahmaroutu) ([#75805](https://github.com/kubernetes/kubernetes/pull/75805), [@brahmaroutu](https://github.com/brahmaroutu)) 1698 - Add `container_sockets`, `container_threads`, and `container_threads_max` metrics ([#81972](https://github.com/kubernetes/kubernetes/pull/81972), [@dashpole](https://github.com/dashpole)) 1699 - Add `container_state` label to `running_container_count` kubelet metrics, to get count of containers based on their state(running/exited/created/unknown) ([#81573](https://github.com/kubernetes/kubernetes/pull/81573), [@irajdeep](https://github.com/irajdeep)) 1700 - Added metric `apiserver_watch_events_total` that can be used to understand the number of watch events in the system. ([#78732](https://github.com/kubernetes/kubernetes/pull/78732), [@mborsz](https://github.com/mborsz)) 1701 - Added metric `apiserver_watch_events_sizes` that can be used to estimate sizes of watch events in the system. ([#80477](https://github.com/kubernetes/kubernetes/pull/80477), [@mborsz](https://github.com/mborsz)) 1702 - Added a new Prometheus counter metric `sync_proxy_rules_iptables_restore_failures_total` for kube-proxy iptables-restore failures (both ipvs and iptables modes) 1703 ([#81210](https://github.com/kubernetes/kubernetes/pull/81210), [@figo](https://github.com/figo)) 1704 - kubelet now exports an `kubelet_evictions` metric that counts the number of pod evictions carried out by the kubelet to reclaim resources ([#81377](https://github.com/kubernetes/kubernetes/pull/81377), [@sjenning](https://github.com/sjenning)) 1705 1706 ### Removed metrics 1707 1708 - Removed cadvisor metric labels `pod_name` and `container_name` to match instrumentation guidelines. Any Prometheus queries that match `pod_name` and `container_name` labels (e.g. cadvisor or kubelet probe metrics) must be updated to use `pod` and `container` instead. ([#80376](https://github.com/kubernetes/kubernetes/pull/80376), [@ehashman](https://github.com/ehashman)) 1709 1710 ### Deprecated/changed metrics 1711 1712 - kube-controller-manager and cloud-controller-manager metrics are now marked as with the ALPHA stability level. ([#81624](https://github.com/kubernetes/kubernetes/pull/81624), [@logicalhan](https://github.com/logicalhan)) 1713 - kube-proxy metrics are now marked as with the ALPHA stability level. ([#81626](https://github.com/kubernetes/kubernetes/pull/81626), [@logicalhan](https://github.com/logicalhan)) 1714 - kube-apiserver metrics are now marked as with the ALPHA stability level. ([#81531](https://github.com/kubernetes/kubernetes/pull/81531), [@logicalhan](https://github.com/logicalhan)) 1715 - kubelet metrics for /metrics and /metrics/probes are now marked as with the ALPHA stability level. ([#81534](https://github.com/kubernetes/kubernetes/pull/81534), [@logicalhan](https://github.com/logicalhan)) 1716 - Scheduler metrics are now marked as with the ALPHA stability level. ([#81576](https://github.com/kubernetes/kubernetes/pull/81576), [@logicalhan](https://github.com/logicalhan)) 1717 - The `rejected` label in `apiserver_admission_webhook_admission_duration_seconds` metrics now properly indicates if the request was rejected. ([#81399](https://github.com/kubernetes/kubernetes/pull/81399), [@roycaihw](https://github.com/roycaihw)) 1718 - Fixed a bug in the CSI metrics that does not return not supported error when a CSI driver does not support metrics. ([#79851](https://github.com/kubernetes/kubernetes/pull/79851), [@jparklab](https://github.com/jparklab)) 1719 - Fix disk stats in LXD using ZFS storage pool and CRI-O missing network metris bug ([#81972](https://github.com/kubernetes/kubernetes/pull/81972), [@dashpole](https://github.com/dashpole)) 1720 1721 ## Notable Features 1722 1723 ### Beta 1724 1725 - Promote WatchBookmark feature to beta and enable it by default. 1726 With WatchBookmark feature, clients are able to request watch events with BOOKMARK type. Clients should not assume bookmarks are returned at any specific interval, nor may they assume the server will send any BOOKMARK event during a session. ([#79786](https://github.com/kubernetes/kubernetes/pull/79786), [@wojtek-t](https://github.com/wojtek-t)) 1727 - The server-side apply feature is now beta ([#81956](https://github.com/kubernetes/kubernetes/pull/81956), [@apelisse](https://github.com/apelisse)) 1728 - Server-side apply will now use the openapi provided in the CRD validation field to help figure out how to correctly merge objects and update ownership. ([#77354](https://github.com/kubernetes/kubernetes/pull/77354), [@jennybuckley](https://github.com/jennybuckley)) 1729 - The `CustomResourceDefaulting` feature is promoted to beta and enabled by default. Defaults may be specified in structural schemas via the `apiextensions.k8s.io/v1` API. See https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#specifying-a-structural-schema for details. ([#81872](https://github.com/kubernetes/kubernetes/pull/81872), [@sttts](https://github.com/sttts)) 1730 - Finalizer Protection for Service LoadBalancers is now in beta (enabled by default). This feature ensures the Service resource is not fully deleted until the correlating load balancer resources are deleted. ([#81691](https://github.com/kubernetes/kubernetes/pull/81691), [@MrHohn](https://github.com/MrHohn)) 1731 - Graduating Windows GMSA support from alpha to beta ([#82110](https://github.com/kubernetes/kubernetes/pull/82110), [@wk8](https://github.com/wk8)) 1732 1733 ### Alpha 1734 1735 - Introduce a new admission controller for RuntimeClass. Initially, RuntimeClass will be used to apply the pod overhead associated with a given RuntimeClass to the Pod `spec` if a corresponding RuntimeClassName is specified. PodOverhead is an alpha feature as of Kubernetes 1.16. ([#78484](https://github.com/kubernetes/kubernetes/pull/78484), [@egernst](https://github.com/egernst)) 1736 - Introduction of the pod overhead feature to the scheduler. This functionality is alpha-level as of 1737 Kubernetes v1.16, and is only honored by servers that enable the PodOverhead feature.gate. ([#78319](https://github.com/kubernetes/kubernetes/pull/78319), [@egernst](https://github.com/egernst)) 1738 - Ephemeral containers have been added in alpha. These temporary containers can be added to running pods for purposes such as debugging, similar to how `kubectl exec` runs a process in an existing container. Also like `kubectl exec`, no resources are reserved for ephemeral containers and they are not restarted when they exit. Note that container namespace targeting is not yet implemented, so [process namespace sharing](https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/) must be enabled to view process from other containers in the pod. ([#59484](https://github.com/kubernetes/kubernetes/pull/59484), [@verb](https://github.com/verb)) 1739 - Pod spread constraints have been added in alpha. You can use these constraints to control how Pods are spread across the cluster among failure-domains. ([#77327](https://github.com/kubernetes/kubernetes/pull/77327), [#77760](https://github.com/kubernetes/kubernetes/pull/77760), [#77828](https://github.com/kubernetes/kubernetes/pull/77828), [#79062](https://github.com/kubernetes/kubernetes/pull/79062), [#80011](https://github.com/kubernetes/kubernetes/pull/80011), [#81068](https://github.com/kubernetes/kubernetes/pull/81068), [@Huang-Wei](https://github.com/Huang-Wei)) 1740 1741 ### CLI Improvements 1742 1743 - the new flag `--endpoint-updates-batch-period` in kube-controller-manager can be used to reduce the number of endpoints updates generated by pod changes. ([#80509](https://github.com/kubernetes/kubernetes/pull/80509), [@mborsz](https://github.com/mborsz)) 1744 - the kubectl `--all-namespaces` flag is now honored by `kubectl wait` ([#81468](https://github.com/kubernetes/kubernetes/pull/81468), [@ashutoshgngwr](https://github.com/ashutoshgngwr)) 1745 - `kubectl get -w` now takes an `--output-watch-events` flag to indicate the event type (ADDED, MODIFIED, DELETED) ([#72416](https://github.com/kubernetes/kubernetes/pull/72416), [@liggitt](https://github.com/liggitt)) 1746 - Adds Endpoint Slice support for kubectl when discovery API group is enabled. ([#81795](https://github.com/kubernetes/kubernetes/pull/81795), [@robscott](https://github.com/robscott)) 1747 1748 ### Misc 1749 1750 - Add `--shutdown-delay-duration` to kube-apiserver in order to delay a graceful shutdown. `/healthz` will keep returning success during this time and requests are normally served, but `/readyz` will return failure immediately. This delay can be used to allow the SDN to update iptables on all nodes and stop sending traffic. ([#74416](https://github.com/kubernetes/kubernetes/pull/74416), [@sttts](https://github.com/sttts)) 1751 Kubeadm now seamlessly migrates the CoreDNS Configuration when upgrading CoreDNS. ([#78033](https://github.com/kubernetes/kubernetes/pull/78033), [@rajansandeep](https://github.com/rajansandeep)) 1752 - Add Endpoint Slice Controller for managing new EndpointSlice resource, disabled by default. ([#81048](https://github.com/kubernetes/kubernetes/pull/81048), [@robscott](https://github.com/robscott)) 1753 - Adds `\livez` for liveness health checking for kube-apiserver. Using the parameter `--maximum-startup-sequence-duration` will allow the liveness endpoint to defer boot-sequence failures for the specified duration period. ([#81969](https://github.com/kubernetes/kubernetes/pull/81969), [@logicalhan](https://github.com/logicalhan)) 1754 - Adds EndpointSlice integration to kube-proxy, can be enabled with EndpointSlice feature gate. ([#81430](https://github.com/kubernetes/kubernetes/pull/81430), [@robscott](https://github.com/robscott)) 1755 - Add status condition to namespace resource ([#73405](https://github.com/kubernetes/kubernetes/pull/73405), [@wozniakjan](https://github.com/wozniakjan)) 1756 - Enhance Azure cloud provider code to support both AAD and ADFS authentication. ([#80841](https://github.com/kubernetes/kubernetes/pull/80841), [@rjaini](https://github.com/rjaini)) 1757 - kubeadm: implement support for concurrent add/remove of stacked etcd members ([#79677](https://github.com/kubernetes/kubernetes/pull/79677), [@neolit123](https://github.com/neolit123)) 1758 - kubeadm: support any Linux kernel version newer than 3.10 ([#81623](https://github.com/kubernetes/kubernetes/pull/81623), [@neolit123](https://github.com/neolit123)) 1759 - Volume expansion is enabled in the default GCE storageclass ([#78672](https://github.com/kubernetes/kubernetes/pull/78672), [@msau42](https://github.com/msau42)) 1760 - kubeadm ClusterConfiguration now supports featureGates: IPv6DualStack: true ([#80145](https://github.com/kubernetes/kubernetes/pull/80145), [@Arvinderpal](https://github.com/Arvinderpal)) 1761 - In order to enable dual-stack support within kubeadm and kubernetes components, as part of the init config file, the user should set feature-gate `IPv6DualStack=true` in the ClusterConfiguration. Additionally, for each worker node, the user should set the feature-gate for kubelet using either `nodeRegistration.kubeletExtraArgs` or `KUBELET_EXTRA_ARGS`. ([#80531](https://github.com/kubernetes/kubernetes/pull/80531), [@Arvinderpal](https://github.com/Arvinderpal)) 1762 - Add possibility to configure controller manager to use IPv6 dual stack: 1763 use `--cluster-cidr="<cidr1>,<cidr2>"`. 1764 Notes: 1765 1. Only the first two CIDRs are used (soft limits for Alpha, might be lifted later on). 1766 2. Only the "RangeAllocator" (default) is allowed as a value for `--cidr-allocator-type`. Cloud allocators are not compatible with IPv6 dual stack 1767 ([#73977](https://github.com/kubernetes/kubernetes/pull/73977), [@khenidak](https://github.com/khenidak)) 1768 - Add scheduling support for RuntimeClasses. RuntimeClasses can now specify nodeSelector constraints & tolerations, which are merged into the PodSpec for pods using that RuntimeClass. ([#80825](https://github.com/kubernetes/kubernetes/pull/80825), [@tallclair](https://github.com/tallclair)) 1769 - When specifying `--(kube|system)-reserved-cgroup`, with `--cgroup-driver=systemd`, it is now possible to use the fully qualified cgroupfs name (i.e. `/test-cgroup.slice`). ([#78793](https://github.com/kubernetes/kubernetes/pull/78793), [@mattjmcnaughton](https://github.com/mattjmcnaughton)) 1770 - Adds support for vSphere volumes on Windows ([#80911](https://github.com/kubernetes/kubernetes/pull/80911), [@gab-satchi](https://github.com/gab-satchi)) 1771 1772 ## API Changes 1773 1774 - The `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration` APIs have been promoted to `admissionregistration.k8s.io/v1`: 1775 - `failurePolicy` default changed from `Ignore` to `Fail` for v1 1776 - `matchPolicy` default changed from `Exact` to `Equivalent` for v1 1777 - `timeout` default changed from `30s` to `10s` for v1 1778 - `sideEffects` default value is removed, and the field made required, and only `None` and `NoneOnDryRun` are permitted for v1 1779 - `admissionReviewVersions` default value is removed and the field made required for v1 (supported versions for AdmissionReview are `v1` and `v1beta1`) 1780 - The `name` field for specified webhooks must be unique for `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration` objects created via `admissionregistration.k8s.io/v1` 1781 - The `AdmissionReview` API sent to and received from admission webhooks has been promoted to `admission.k8s.io/v1`. Webhooks can specify a preference for receiving `v1` AdmissionReview objects with `admissionReviewVersions: ["v1","v1beta1"]`, and must respond with an API object in the same `apiVersion` they are sent. When webhooks use `admission.k8s.io/v1`, the following additional validation is performed on their responses: 1782 - `response.patch` and `response.patchType` are not permitted from validating admission webhooks 1783 - `apiVersion: "admission.k8s.io/v1"` is required 1784 - `kind: "AdmissionReview"` is required 1785 - `response.uid: "<value of request.uid>"` is required 1786 - `response.patchType: "JSONPatch"` is required (if `response.patch` is set) ([#80231](https://github.com/kubernetes/kubernetes/pull/80231), [@liggitt](https://github.com/liggitt)) 1787 - The `CustomResourceDefinition` API type is promoted to `apiextensions.k8s.io/v1` with the following changes: 1788 - Use of the new `default` feature in validation schemas is limited to v1 1789 - `spec.scope` is no longer defaulted to `Namespaced` and must be explicitly specified 1790 - `spec.version` is removed in v1; use `spec.versions` instead 1791 - `spec.validation` is removed in v1; use `spec.versions[*].schema` instead 1792 - `spec.subresources` is removed in v1; use `spec.versions[*].subresources` instead 1793 - `spec.additionalPrinterColumns` is removed in v1; use `spec.versions[*].additionalPrinterColumns` instead 1794 - `spec.conversion.webhookClientConfig` is moved to `spec.conversion.webhook.clientConfig` in v1 1795 - `spec.conversion.conversionReviewVersions` is moved to `spec.conversion.webhook.conversionReviewVersions` in v1 1796 - `spec.versions[*].schema.openAPIV3Schema` is now required when creating v1 CustomResourceDefinitions 1797 - `spec.preserveUnknownFields: true` is disallowed when creating v1 CustomResourceDefinitions; it must be specified within schema definitions as `x-kubernetes-preserve-unknown-fields: true` 1798 - In `additionalPrinterColumns` items, the `JSONPath` field was renamed to `jsonPath` in v1 (fixes https://github.com/kubernetes/kubernetes/issues/66531) 1799 The `apiextensions.k8s.io/v1beta1` version of `CustomResourceDefinition` is deprecated and will no longer be served in v1.19. ([#79604](https://github.com/kubernetes/kubernetes/pull/79604), [@liggitt](https://github.com/liggitt)) 1800 - The `ConversionReview` API sent to and received from custom resource CustomResourceDefinition conversion webhooks has been promoted to `apiextensions.k8s.io/v1`. CustomResourceDefinition conversion webhooks can now indicate they support receiving and responding with `ConversionReview` API objects in the `apiextensions.k8s.io/v1` version by including `v1` in the `conversionReviewVersions` list in their CustomResourceDefinition. Conversion webhooks must respond with a ConversionReview object in the same apiVersion they receive. `apiextensions.k8s.io/v1` `ConversionReview` responses must specify a `response.uid` that matches the `request.uid` of the object they were sent. ([#81476](https://github.com/kubernetes/kubernetes/pull/81476), [@liggitt](https://github.com/liggitt)) 1801 - Add scheduling support for RuntimeClasses. RuntimeClasses can now specify nodeSelector constraints & tolerations, which are merged into the PodSpec for pods using that RuntimeClass. ([#80825](https://github.com/kubernetes/kubernetes/pull/80825), [@tallclair](https://github.com/tallclair)) 1802 - Kubelet should now more reliably report the same primary node IP even if the set of node IPs reported by the CloudProvider changes. ([#79391](https://github.com/kubernetes/kubernetes/pull/79391), [@danwinship](https://github.com/danwinship)) 1803 - Omit nil or empty field when calculating container hash value to avoid hash changed. For a new field with a non-nil default value in the container spec, the hash would still get changed. ([#57741](https://github.com/kubernetes/kubernetes/pull/57741), [@dixudx](https://github.com/dixudx)) 1804 - Property `conditions` in `apiextensions.v1beta1.CustomResourceDefinitionStatus` and `apiextensions.v1.CustomResourceDefinitionStatus` is now optional instead of required. ([#64996](https://github.com/kubernetes/kubernetes/pull/64996), [@roycaihw](https://github.com/roycaihw)) 1805 - When the status of a CustomResourceDefinition condition changes, its corresponding `lastTransitionTime` is now updated. ([#69655](https://github.com/kubernetes/kubernetes/pull/69655), [@CaoShuFeng](https://github.com/CaoShuFeng)) 1806 1807 ## Other notable changes 1808 1809 ### API Machinery 1810 1811 - Remove `GetReference()` and `GetPartialReference()` function from `pkg/api/ref`, as the same function exists also in `staging/src/k8s.io/client-go/tools/ref` ([#80361](https://github.com/kubernetes/kubernetes/pull/80361), [@wojtek-t](https://github.com/wojtek-t)) 1812 - Verify that CRD default values in OpenAPI specs are pruned, with the exceptions of values under `metadata`. ([#78829](https://github.com/kubernetes/kubernetes/pull/78829), [@sttts](https://github.com/sttts)) 1813 - Fixes a bug that when there is a "connection refused" error, the reflector's ListAndWatch func will return directly but what expected is that sleep 1 second and rewatch since the specified resourceVersion. 1814 ([#81634](https://github.com/kubernetes/kubernetes/pull/81634), [@likakuli](https://github.com/likakuli)) 1815 - Resolves an issue serving aggregated APIs backed by services that respond to requests to `/` with non-2xx HTTP responses ([#79895](https://github.com/kubernetes/kubernetes/pull/79895), [@deads2k](https://github.com/deads2k)) 1816 - The CRD handler now properly re-creates stale CR storage to reflect CRD update. ([#79114](https://github.com/kubernetes/kubernetes/pull/79114), [@roycaihw](https://github.com/roycaihw)) 1817 - Fix CVE-2019-11247: API server allows access to custom resources via wrong scope ([#80750](https://github.com/kubernetes/kubernetes/pull/80750), [@sttts](https://github.com/sttts)) 1818 - Fixed a bug with the openAPI definition for `io.k8s.apimachinery.pkg.runtime.RawExtension`, which previously required a field `raw` to be specified ([#80773](https://github.com/kubernetes/kubernetes/pull/80773), [@jennybuckley](https://github.com/jennybuckley)) 1819 - Property `conditions` in `apiextensions.v1beta1.CustomResourceDefinitionStatus` and `apiextensions.v1.CustomResourceDefinitionStatus` is now optional instead of required. ([#64996](https://github.com/kubernetes/kubernetes/pull/64996), [@roycaihw](https://github.com/roycaihw)) 1820 - Resolves a transient 404 response to custom resource requests during server startup ([#81244](https://github.com/kubernetes/kubernetes/pull/81244), [@liggitt](https://github.com/liggitt)) 1821 - OpenAPI now advertises correctly supported patch types for custom resources ([#81515](https://github.com/kubernetes/kubernetes/pull/81515), [@liggitt](https://github.com/liggitt)) 1822 - When the status of a CRD Condition changes, it's corresponding `LastTransitionTime` is now updated. ([#69655](https://github.com/kubernetes/kubernetes/pull/69655), [@CaoShuFeng](https://github.com/CaoShuFeng)) 1823 - Add `metadata.generation=1` to old CustomResources. ([#82005](https://github.com/kubernetes/kubernetes/pull/82005), [@sttts](https://github.com/sttts)) 1824 - Fix a bug in the apiserver that could cause a valid update request to be rejected with a precondition check failure. ([#82303](https://github.com/kubernetes/kubernetes/pull/82303), [@roycaihw](https://github.com/roycaihw)) 1825 - Fixes regression in logging spurious stack traces when proxied connections are closed by the backend ([#82588](https://github.com/kubernetes/kubernetes/pull/82588), [@liggitt](https://github.com/liggitt)) 1826 - RateLimiter add a context-aware method, fix client-go request goruntine backlog in async timeout scene. ([#79375](https://github.com/kubernetes/kubernetes/pull/79375), [@answer1991](https://github.com/answer1991)) 1827 - Add a `Patch` method to `ScaleInterface` ([#80699](https://github.com/kubernetes/kubernetes/pull/80699), [@knight42](https://github.com/knight42)) 1828 - CRDs under k8s.io and kubernetes.io must have the `api-approved.kubernetes.io` set to either `unapproved.*` or a link to the pull request approving the schema. See https://github.com/kubernetes/enhancements/pull/1111 for more details. ([#79992](https://github.com/kubernetes/kubernetes/pull/79992), [@deads2k](https://github.com/deads2k)) 1829 - KMS Providers will install a healthz check for the status of kms-plugin in kube-apiservers' encryption config. ([#78540](https://github.com/kubernetes/kubernetes/pull/78540), [@immutableT](https://github.com/immutableT)) 1830 - Improves validation errors for custom resources ([#81212](https://github.com/kubernetes/kubernetes/pull/81212), [@liggitt](https://github.com/liggitt)) 1831 - Populate object name for admission attributes when CREATE ([#53185](https://github.com/kubernetes/kubernetes/pull/53185), [@dixudx](https://github.com/dixudx)) 1832 - Add Overhead field to the PodSpec and RuntimeClass types as part of the Pod Overhead KEP ([#76968](https://github.com/kubernetes/kubernetes/pull/76968), [@egernst](https://github.com/egernst)) 1833 1834 ### Apps 1835 1836 - Fix a bug that pods not be deleted from unmatched nodes by daemon controller ([#78974](https://github.com/kubernetes/kubernetes/pull/78974), [@DaiHao](https://github.com/DaiHao)) 1837 - Fix a bug that causes DaemonSet rolling update hang when there exist failed pods. ([#78170](https://github.com/kubernetes/kubernetes/pull/78170), [@DaiHao](https://github.com/DaiHao)) 1838 1839 ### Auth 1840 1841 - Service account tokens now include the JWT Key ID field in their header. ([#78502](https://github.com/kubernetes/kubernetes/pull/78502), [@ahmedtd](https://github.com/ahmedtd)) 1842 - The nbf (not before) claim, if present in ID token, is now enforced. ([#81413](https://github.com/kubernetes/kubernetes/pull/81413), [@anderseknert](https://github.com/anderseknert)) 1843 1844 ### CLI 1845 1846 - Fix CVE-2019-11249: Incomplete fixes for CVE-2019-1002101 and CVE-2019-11246, kubectl cp potential directory traversal ([#80436](https://github.com/kubernetes/kubernetes/pull/80436), [@M00nF1sh](https://github.com/M00nF1sh)) 1847 - Fix the bash completion error with override flags. ([#80802](https://github.com/kubernetes/kubernetes/pull/80802), [@dtaniwaki](https://github.com/dtaniwaki)) 1848 - Fix a bug in server printer that could cause kube-apiserver to panic. ([#79349](https://github.com/kubernetes/kubernetes/pull/79349), [@roycaihw](https://github.com/roycaihw)) 1849 - Fix invalid "time stamp is the future" error when kubectl cp-ing a file ([#73982](https://github.com/kubernetes/kubernetes/pull/73982), [@tanshanshan](https://github.com/tanshanshan)) 1850 - Fix a bug where `kubectl set config` hangs and uses 100% CPU on some invalid property names ([#79000](https://github.com/kubernetes/kubernetes/pull/79000), [@pswica](https://github.com/pswica)) 1851 - Fix output of `kubectl get --watch-only` when watching a single resource ([#79345](https://github.com/kubernetes/kubernetes/pull/79345), [@liggitt](https://github.com/liggitt)) 1852 - Make kubectl get `--ignore-not-found` continue processing when encountering error. ([#82120](https://github.com/kubernetes/kubernetes/pull/82120), [@soltysh](https://github.com/soltysh)) 1853 - Correct a reference to a not/no longer used kustomize subcommand in the documentation ([#82535](https://github.com/kubernetes/kubernetes/pull/82535), [@demobox](https://github.com/demobox)) 1854 - kubectl could scale custom resource again ([#81342](https://github.com/kubernetes/kubernetes/pull/81342), [@knight42](https://github.com/knight42)) 1855 - Add PodOverhead awareness to kubectl ([#81929](https://github.com/kubernetes/kubernetes/pull/81929), [@egernst](https://github.com/egernst)) 1856 1857 ### Cloud Provider 1858 1859 - When a load balancer type service is created in a k8s cluster that is backed by Azure Standard Load Balancer, the corresponding load balancer rule added in the Azure Standard Load Balancer would now have the "EnableTcpReset" property set to true. ([#80624](https://github.com/kubernetes/kubernetes/pull/80624), [@xuto2](https://github.com/xuto2)) 1860 - Switch to VM Update call in attach/detach disk operation, original CreateOrUpdate call may lead to orphaned VMs or blocked resources ([#81208](https://github.com/kubernetes/kubernetes/pull/81208), [@andyzhangx](https://github.com/andyzhangx)) 1861 - Fix azure disk naming matching issue due to case sensitive comparison ([#81720](https://github.com/kubernetes/kubernetes/pull/81720), [@andyzhangx](https://github.com/andyzhangx)) 1862 - Fix retry issues when the nodes are under deleting on Azure ([#80419](https://github.com/kubernetes/kubernetes/pull/80419), [@feiskyer](https://github.com/feiskyer)) 1863 - Fix conflicted cache when the requests are canceled by other Azure operations. ([#81282](https://github.com/kubernetes/kubernetes/pull/81282), [@feiskyer](https://github.com/feiskyer)) 1864 - Fix make azure disk URI as case insensitive ([#79020](https://github.com/kubernetes/kubernetes/pull/79020), [@andyzhangx](https://github.com/andyzhangx)) 1865 - Fix VMSS LoadBalancer backend pools so that the network won't be broken when instances are upgraded to latest model ([#81411](https://github.com/kubernetes/kubernetes/pull/81411), [@nilo19](https://github.com/nilo19)) 1866 - Default resourceGroup should be used when the value of annotation azure-load-balancer-resource-group is an empty string. ([#79514](https://github.com/kubernetes/kubernetes/pull/79514), [@feiskyer](https://github.com/feiskyer)) 1867 - Kubelet could be run with no Azure identity without subscriptionId configured now. 1868 A sample cloud provider configure is: '{"vmType": "vmss", "useInstanceMetadata": true}'. ([#81500](https://github.com/kubernetes/kubernetes/pull/81500), [@feiskyer](https://github.com/feiskyer)) 1869 - Fix public IP not found issues for VMSS nodes ([#80703](https://github.com/kubernetes/kubernetes/pull/80703), [@feiskyer](https://github.com/feiskyer)) 1870 - Fix Azure client requests stuck issues on http.StatusTooManyRequests (HTTP Code 429). ([#81279](https://github.com/kubernetes/kubernetes/pull/81279), [@feiskyer](https://github.com/feiskyer)) 1871 - Add a service annotation `service.beta.kubernetes.io/azure-pip-name` to specify the public IP name for Azure load balancer. ([#81213](https://github.com/kubernetes/kubernetes/pull/81213), [@nilo19](https://github.com/nilo19)) 1872 - Optimize EC2 DescribeInstances API calls in aws cloud provider library by querying instance ID instead of EC2 filters when possible ([#78140](https://github.com/kubernetes/kubernetes/pull/78140), [@zhan849](https://github.com/zhan849)) 1873 - Creates an annotation `service.beta.kubernetes.io/aws-load-balancer-eip-allocations` to assign AWS EIP to the newly created Network Load Balancer. Number of allocations and subnets must match. ([#69263](https://github.com/kubernetes/kubernetes/pull/69263), [@brooksgarrett](https://github.com/brooksgarrett)) 1874 - Add an azure cloud configuration `LoadBalancerName` and `LoadBalancerResourceGroup` to allow the corresponding customizations of azure load balancer. ([#81054](https://github.com/kubernetes/kubernetes/pull/81054), [@nilo19](https://github.com/nilo19)) 1875 1876 ### Cluster Lifecycle 1877 1878 - Fix error handling and potential go null pointer exception in kubeadm upgrade diff ([#80648](https://github.com/kubernetes/kubernetes/pull/80648), [@odinuge](https://github.com/odinuge)) 1879 - kubeadm: fall back to client version in case of certain HTTP errors ([#80024](https://github.com/kubernetes/kubernetes/pull/80024), [@RainbowMango](https://github.com/RainbowMango)) 1880 - kubeadm: fix a potential panic if kubeadm discovers an invalid, existing kubeconfig file ([#79165](https://github.com/kubernetes/kubernetes/pull/79165), [@neolit123](https://github.com/neolit123)) 1881 - kubeadm: treat non-fatal errors as warnings when doing reset ([#80862](https://github.com/kubernetes/kubernetes/pull/80862), [@drpaneas](https://github.com/drpaneas)) 1882 - kubeadm: prevent PSP blocking of upgrade image prepull by using a non-root user ([#77792](https://github.com/kubernetes/kubernetes/pull/77792), [@neolit123](https://github.com/neolit123)) 1883 - kubeadm: fix "certificate-authority" files not being pre-loaded when using file discovery ([#80966](https://github.com/kubernetes/kubernetes/pull/80966), [@neolit123](https://github.com/neolit123)) 1884 - Add instruction to setup "Application Default Credentials" to run GCE Windows e2e tests locally. ([#81337](https://github.com/kubernetes/kubernetes/pull/81337), [@YangLu1031](https://github.com/YangLu1031)) 1885 - Fix error in `kubeadm join --discovery-file` when using discovery files with embedded credentials ([#80675](https://github.com/kubernetes/kubernetes/pull/80675), [@fabriziopandini](https://github.com/fabriziopandini)) 1886 - Fix remove the etcd member from the cluster during a kubeadm reset. ([#79326](https://github.com/kubernetes/kubernetes/pull/79326), [@bradbeam](https://github.com/bradbeam)) 1887 - kubeadm: the permissions of generated CSR files are changed from 0644 to 0600 ([#81217](https://github.com/kubernetes/kubernetes/pull/81217), [@SataQiu](https://github.com/SataQiu)) 1888 - kubeadm: avoid double deletion of the upgrade prepull DaemonSet ([#80798](https://github.com/kubernetes/kubernetes/pull/80798), [@xlgao-zju](https://github.com/xlgao-zju)) 1889 - kubeadm: introduce deterministic ordering for the certificates generation in the phase command `kubeadm init phase certs`. ([#78556](https://github.com/kubernetes/kubernetes/pull/78556), [@neolit123](https://github.com/neolit123)) 1890 - kubeadm: implement retry logic for certain ConfigMap failures when joining nodes ([#78915](https://github.com/kubernetes/kubernetes/pull/78915), [@ereslibre](https://github.com/ereslibre)) 1891 - kubeadm: use etcd's /health endpoint for a HTTP liveness probe on localhost instead of having a custom health check using etcdctl ([#81385](https://github.com/kubernetes/kubernetes/pull/81385), [@neolit123](https://github.com/neolit123)) 1892 - kubeadm reset: unmount directories under `/var/lib/kubelet` for Linux only ([#81494](https://github.com/kubernetes/kubernetes/pull/81494), [@Klaven](https://github.com/Klaven)) 1893 - kubeadm: fix the bug that `--cri-socket` flag does not work for `kubeadm reset` ([#79498](https://github.com/kubernetes/kubernetes/pull/79498), [@SataQiu](https://github.com/SataQiu)) 1894 - kubeadm: produce errors if they occur when resetting cluster status for a control-plane node ([#80573](https://github.com/kubernetes/kubernetes/pull/80573), [@bart0sh](https://github.com/bart0sh)) 1895 - Fix an error when using external etcd but storing etcd certificates in the same folder with the same name used by kubeadm for local etcd certificates; for an older version of kubeadm, the workaround is to avoid file name used by kubeadm for local etcd. ([#80867](https://github.com/kubernetes/kubernetes/pull/80867), [@fabriziopandini](https://github.com/fabriziopandini)) 1896 - `kubeadm join` fails if file-based discovery is too long, with a default timeout of 5 minutes. ([#80804](https://github.com/kubernetes/kubernetes/pull/80804), [@olivierlemasle](https://github.com/olivierlemasle)) 1897 - kubeadm: fixed ignoring errors when pulling control plane images ([#80529](https://github.com/kubernetes/kubernetes/pull/80529), [@bart0sh](https://github.com/bart0sh)) 1898 - Fix a bug in kube-addon-manager's leader election logic that made all replicas active. ([#80575](https://github.com/kubernetes/kubernetes/pull/80575), [@mborsz](https://github.com/mborsz)) 1899 - kubeadm: prevent overriding of certain kubelet security configuration parameters if the user wished to modify them ([#81903](https://github.com/kubernetes/kubernetes/pull/81903), [@jfbai](https://github.com/jfbai)) 1900 - kubeadm no longer performs IPVS checks as part of its preflight checks ([#81791](https://github.com/kubernetes/kubernetes/pull/81791), [@yastij](https://github.com/yastij)) 1901 - kubeadm: fix for HTTPProxy check for IPv6 addresses ([#82267](https://github.com/kubernetes/kubernetes/pull/82267), [@kad](https://github.com/kad)) 1902 - kubeadm: Allow users to skip the kube-proxy init addon phase during init and still be able to join a cluster and perform some other minor operations (but not upgrade). ([#82248](https://github.com/kubernetes/kubernetes/pull/82248), [@rosti](https://github.com/rosti)) 1903 - Mounts `/home/kubernetes/bin/nvidia/vulkan/icd.d` on the host to `/etc/vulkan/icd.d` inside containers requesting GPU. ([#78868](https://github.com/kubernetes/kubernetes/pull/78868), [@chardch](https://github.com/chardch)) 1904 - kubeadm: use the `--pod-network-cidr` flag to init or use the podSubnet field in the kubeadm config to pass a comma separated list of pod CIDRs. ([#79033](https://github.com/kubernetes/kubernetes/pull/79033), [@Arvinderpal](https://github.com/Arvinderpal)) 1905 - kubeadm: provide `--control-plane-endpoint` flag for `controlPlaneEndpoint` ([#79270](https://github.com/kubernetes/kubernetes/pull/79270), [@SataQiu](https://github.com/SataQiu)) 1906 - kubeadm: enable secure serving for the kube-scheduler ([#80951](https://github.com/kubernetes/kubernetes/pull/80951), [@neolit123](https://github.com/neolit123)) 1907 - kubeadm: print the stack trace of an error for klog level `--v>=5` ([#80937](https://github.com/kubernetes/kubernetes/pull/80937), [@neolit123](https://github.com/neolit123)) 1908 - Add `--kubernetes-version` to `kubeadm init phase certs ca` and `kubeadm init phase kubeconfig` ([#80115](https://github.com/kubernetes/kubernetes/pull/80115), [@gyuho](https://github.com/gyuho)) 1909 - kubeadm: support fetching configuration from the original cluster for `upgrade diff` ([#80025](https://github.com/kubernetes/kubernetes/pull/80025), [@SataQiu](https://github.com/SataQiu)) 1910 - When using the conformance test image, a new environment variable `E2E_USE_GO_RUNNER` will cause the tests to be run with the new golang-based test runner rather than the current bash wrapper. ([#79284](https://github.com/kubernetes/kubernetes/pull/79284), [@johnSchnake](https://github.com/johnSchnake)) 1911 - Implement a new feature that allows applying kustomize patches to static pod manifests generated by kubeadm. ([#80905](https://github.com/kubernetes/kubernetes/pull/80905), [@fabriziopandini](https://github.com/fabriziopandini)) 1912 - The 404 request handler for the GCE Ingress load balancer now exports prometheus metrics, including: 1913 1914 - `http_404_request_total` (the number of 404 requests handled) 1915 - `http_404_request_duration_ms` (the amount of time the server took to respond in ms) 1916 1917 Also includes percentile groupings. The directory for the default 404 handler includes instructions on how to enable prometheus for monitoring and setting alerts. 1918 ([#79106](https://github.com/kubernetes/kubernetes/pull/79106), [@vbannai](https://github.com/vbannai)) 1919 1920 ### Instrumentation 1921 1922 - Kibana has been slightly revamped/improved in the latest version ([#80421](https://github.com/kubernetes/kubernetes/pull/80421), [@lostick](https://github.com/lostick)) 1923 1924 ### Network 1925 1926 - Fix a string comparison bug in IPVS graceful termination where UDP real servers are not deleted. ([#78999](https://github.com/kubernetes/kubernetes/pull/78999), [@andrewsykim](https://github.com/andrewsykim)) 1927 - `kube-proxy --cleanup will` return the correct exit code if the cleanup was successful ([#78775](https://github.com/kubernetes/kubernetes/pull/78775), [@johscheuer](https://github.com/johscheuer)) 1928 - Fix a bug in the IPVS proxier where virtual servers are not cleaned up even though the corresponding Service object was deleted. ([#80942](https://github.com/kubernetes/kubernetes/pull/80942), [@gongguan](https://github.com/gongguan)) 1929 - kube-proxy waits for some duration for the node to be defined. ([#77167](https://github.com/kubernetes/kubernetes/pull/77167), [@paulsubrata55](https://github.com/paulsubrata55)) 1930 - Increase log level for graceful termination to `v=5` ([#80100](https://github.com/kubernetes/kubernetes/pull/80100), [@andrewsykim](https://github.com/andrewsykim)) 1931 - Reduce kube-proxy CPU usage in IPVS mode when a large number of nodePort services exist. ([#79444](https://github.com/kubernetes/kubernetes/pull/79444), [@cezarsa](https://github.com/cezarsa)) 1932 - Fix in kube-proxy for SCTP nodeport service which only works for node's InternalIP, but doesn't work for other IPs present in the node when ipvs is enabled. ([#81477](https://github.com/kubernetes/kubernetes/pull/81477), [@paulsubrata55](https://github.com/paulsubrata55)) 1933 - Ensure the `KUBE-MARK-DROP` chain in kube-proxy IPVS mode. The chain is ensured for both IPv4 and IPv6 in dual-stack operation. ([#82214](https://github.com/kubernetes/kubernetes/pull/82214), [@uablrek](https://github.com/uablrek)) 1934 - Introduce `node.kubernetes.io/exclude-balancer` and `node.kubernetes.io/exclude-disruption` labels in alpha to prevent cluster deployers from being dependent on the optional `node-role` labels which not all clusters may provide. ([#80238](https://github.com/kubernetes/kubernetes/pull/80238), [@smarterclayton](https://github.com/smarterclayton)) 1935 - If targetPort is changed that will process by service controller ([#77712](https://github.com/kubernetes/kubernetes/pull/77712), [@Sn0rt](https://github.com/Sn0rt)) 1936 1937 ### Node 1938 1939 - Remove PIDs cgroup controller requirement when related feature gates are disabled 1940 ([#79073](https://github.com/kubernetes/kubernetes/pull/79073), [@rafatio](https://github.com/rafatio)) 1941 - Fix kubelet NodeLease potential performance issues. Kubelet now will try to update lease using cached one instead of get from API Server every time. ([#81174](https://github.com/kubernetes/kubernetes/pull/81174), [@answer1991](https://github.com/answer1991)) 1942 - Passing an invalid policy name in the `--cpu-manager-policy` flag will now cause the kubelet to fail instead of simply ignoring the flag and running the `cpumanager`’s default policy instead. ([#80294](https://github.com/kubernetes/kubernetes/pull/80294), [@klueska](https://github.com/klueska)) 1943 - Make node lease renew interval more heuristic based on node-status-update-frequency in kubelet ([#80173](https://github.com/kubernetes/kubernetes/pull/80173), [@gaorong](https://github.com/gaorong)) 1944 - Kubelet should now more reliably report the same primary node IP even if the set of node IPs reported by the CloudProvider changes. ([#79391](https://github.com/kubernetes/kubernetes/pull/79391), [@danwinship](https://github.com/danwinship)) 1945 - Omit `nil` or empty field when calculating container hash value to avoid hash changed. For a new field with a non-nil default value in the container spec, the hash would still get changed. ([#57741](https://github.com/kubernetes/kubernetes/pull/57741), [@dixudx](https://github.com/dixudx)) 1946 - Fix a bug where kubelet would not retry pod sandbox creation when the restart policy of the pod is Never ([#79451](https://github.com/kubernetes/kubernetes/pull/79451), [@yujuhong](https://github.com/yujuhong)) 1947 - Limit the body length of exec readiness/liveness probes. remote CRIs and Docker shim read a max of 16MB output of which the exec probe itself inspects 10kb. ([#82514](https://github.com/kubernetes/kubernetes/pull/82514), [@dims](https://github.com/dims)) 1948 - Single static pod files and pod files from http endpoints cannot be larger than 10 MB. HTTP probe payloads are now truncated to 10KB. ([#82669](https://github.com/kubernetes/kubernetes/pull/82669), [@rphillips](https://github.com/rphillips)) 1949 - Introduce support for applying pod overhead to pod cgroups, if the PodOverhead feature is enabled. ([#79247](https://github.com/kubernetes/kubernetes/pull/79247), [@egernst](https://github.com/egernst)) 1950 - Node-Problem-Detector v0.7.1 is used on GCI ([#80726](https://github.com/kubernetes/kubernetes/pull/80726), [@wangzhen127](https://github.com/wangzhen127)) 1951 - Node-Problem-Detector v0.7.1 is used for addon daemonset. ([#82140](https://github.com/kubernetes/kubernetes/pull/82140), [@wangzhen127](https://github.com/wangzhen127)) 1952 - Enable cAdvisor ProcessMetrics collecting. ([#79002](https://github.com/kubernetes/kubernetes/pull/79002), [@jiayingz](https://github.com/jiayingz)) 1953 - kubelet: change `node-lease-renew-interval` to 0.25 of lease-renew-duration ([#80429](https://github.com/kubernetes/kubernetes/pull/80429), [@gaorong](https://github.com/gaorong)) 1954 - Attempt to set the kubelet's hostname & internal IP if `--cloud-provider=external` and no node addresses exists ([#75229](https://github.com/kubernetes/kubernetes/pull/75229), [@andrewsykim](https://github.com/andrewsykim)) 1955 1956 ### Scheduling 1957 1958 - Scheduler should terminate when it loses leader lock. ([#81306](https://github.com/kubernetes/kubernetes/pull/81306), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) 1959 - If scheduler extender filtered a not found node, current scheduling round for this pod will just be skipped. 1960 ([#79641](https://github.com/kubernetes/kubernetes/pull/79641), [@yqwang-ms](https://github.com/yqwang-ms)) 1961 - Extender bind should respect IsInterested ([#79804](https://github.com/kubernetes/kubernetes/pull/79804), [@yqwang-ms](https://github.com/yqwang-ms)) 1962 - Fix an issue with toleration merging & whitelist checking in the PodTolerationRestriction admission controller. ([#81732](https://github.com/kubernetes/kubernetes/pull/81732), [@tallclair](https://github.com/tallclair)) 1963 - Add a helper function to decode scheduler plugin args. ([#80696](https://github.com/kubernetes/kubernetes/pull/80696), [@hex108](https://github.com/hex108)) 1964 - Fix filter plugins are not been called during preemption ([#81876](https://github.com/kubernetes/kubernetes/pull/81876), [@wgliang](https://github.com/wgliang)) 1965 - Fix an issue that the correct PluginConfig.Args is not passed to the corresponding PluginFactory in kube-scheduler when multiple PluginConfig items are defined. ([#82483](https://github.com/kubernetes/kubernetes/pull/82483), [@everpeace](https://github.com/everpeace)) 1966 - Take the context as the first argument of Schedule. ([#82119](https://github.com/kubernetes/kubernetes/pull/82119), [@wgliang](https://github.com/wgliang)) 1967 - Implement `post-filter` extension point for scheduling framework ([#78097](https://github.com/kubernetes/kubernetes/pull/78097), [@draveness](https://github.com/draveness)) 1968 - Add Bind extension point of the scheduling framework ([#78513](https://github.com/kubernetes/kubernetes/pull/78513), [@chenchun](https://github.com/chenchun)) 1969 - Add Filter extension point to the scheduling framework. ([#78477](https://github.com/kubernetes/kubernetes/pull/78477), [@YoubingLi](https://github.com/YoubingLi)) 1970 - Return error when the scoring plugin returns score out of range `[0, 100]`. ([#81015](https://github.com/kubernetes/kubernetes/pull/81015), [@draveness](https://github.com/draveness)) 1971 - Use a named array instead of a score array in normalizing-score phase. ([#80901](https://github.com/kubernetes/kubernetes/pull/80901), [@draveness](https://github.com/draveness)) 1972 - Updates the `requestedToCapacityRatioArguments` to add resources parameter that allows the users to specify the resource name along with weights for each resource to score nodes based on the request to capacity ratio. ([#77688](https://github.com/kubernetes/kubernetes/pull/77688), [@sudeshsh](https://github.com/sudeshsh)) 1973 - Add `UnschedulableAndUnresolvable` status code for scheduling framework ([#82034](https://github.com/kubernetes/kubernetes/pull/82034), [@alculquicondor](https://github.com/alculquicondor)) 1974 - Add normalize plugin extension point for the scheduling framework. 1975 ([#80383](https://github.com/kubernetes/kubernetes/pull/80383), [@liu-cong](https://github.com/liu-cong)) 1976 - Add Bind extension point to the scheduling framework. ([#79313](https://github.com/kubernetes/kubernetes/pull/79313), [@chenchun](https://github.com/chenchun)) 1977 - Add Score extension point to the scheduling framework. ([#79109](https://github.com/kubernetes/kubernetes/pull/79109), [@ahg-g](https://github.com/ahg-g)) 1978 - Add Pre-filter extension point to the scheduling framework. ([#78005](https://github.com/kubernetes/kubernetes/pull/78005), [@ahg-g](https://github.com/ahg-g)) 1979 - Add support for writing out of tree custom scheduler plugins. ([#78162](https://github.com/kubernetes/kubernetes/pull/78162), [@hex108](https://github.com/hex108)) 1980 1981 ### Storage 1982 1983 - Fix possible file descriptor leak and closing of dirs in `doSafeMakeDir` ([#79534](https://github.com/kubernetes/kubernetes/pull/79534), [@odinuge](https://github.com/odinuge)) 1984 - Azure disks of shared kind will no longer fail if they do not contain `skuname` or `storageaccounttype`. ([#80837](https://github.com/kubernetes/kubernetes/pull/80837), [@rmweir](https://github.com/rmweir)) 1985 - Fix CSI plugin supporting raw block that does not need attach mounted failed ([#79920](https://github.com/kubernetes/kubernetes/pull/79920), [@cwdsuzhou](https://github.com/cwdsuzhou)) 1986 - Reduces GCE PD Node Attach Limits by 1 since the node boot disk is considered an attachable disk ([#80923](https://github.com/kubernetes/kubernetes/pull/80923), [@davidz627](https://github.com/davidz627)) 1987 - Remove iSCSI volume storage cleartext secrets in logs ([#81215](https://github.com/kubernetes/kubernetes/pull/81215), [@zouyee](https://github.com/zouyee)) 1988 - Fixes validation of VolumeAttachment API objects created with inline volume sources. ([#80945](https://github.com/kubernetes/kubernetes/pull/80945), [@tedyu](https://github.com/tedyu)) 1989 - Changes timeout value in csi plugin from 15s to 2min which fixes the timeout issue ([#79529](https://github.com/kubernetes/kubernetes/pull/79529), [@andyzhangx](https://github.com/andyzhangx)) 1990 - Fix kubelet fail to delete orphaned pod directory when the kubelet's pods directory (default is `/var/lib/kubelet/pods`) symbolically links to another disk device's directory ([#79094](https://github.com/kubernetes/kubernetes/pull/79094), [@gaorong](https://github.com/gaorong)) 1991 1992 ## Testing 1993 1994 - Fix pod list return value of `framework.WaitForPodsWithLabelRunningReady` ([#78687](https://github.com/kubernetes/kubernetes/pull/78687), [@pohly](https://github.com/pohly)) 1995 - Adding `TerminationGracePeriodSeconds` to the test framework API ([#82170](https://github.com/kubernetes/kubernetes/pull/82170), [@vivekbagade](https://github.com/vivekbagade)) 1996 - `/test/e2e/framework`: Adds a flag `non-blocking-taints` which allows tests to run in environments with tainted nodes. String value should be a comma-separated list. ([#81043](https://github.com/kubernetes/kubernetes/pull/81043), [@johnSchnake](https://github.com/johnSchnake)) 1997 - Move CSI volume expansion to beta. ([#81467](https://github.com/kubernetes/kubernetes/pull/81467), [@bertinatto](https://github.com/bertinatto)) 1998 - Added E2E tests validating WindowsOptions.RunAsUserName. ([#79539](https://github.com/kubernetes/kubernetes/pull/79539), [@bclau](https://github.com/bclau)) 1999 - `framework.ExpectNoError` no longer logs the error and instead relies on using the new `log.Fail` as gomega fail handler. ([#80253](https://github.com/kubernetes/kubernetes/pull/80253), [@pohly](https://github.com/pohly)) 2000 2001 ### Windows 2002 2003 - On Windows systems, `%USERPROFILE%` is now preferred over `%HOMEDRIVE%\%HOMEPATH%` as the home folder if `%HOMEDRIVE%\%HOMEPATH%` does not contain a `.kube\config` file, and `%USERPROFILE%` exists and is writable. ([#73923](https://github.com/kubernetes/kubernetes/pull/73923), [@liggitt](https://github.com/liggitt)) 2004 - Add support for AWS EBS on windows ([#79552](https://github.com/kubernetes/kubernetes/pull/79552), [@wongma7](https://github.com/wongma7)) 2005 - Support Kubelet plugin watcher on Windows nodes. ([#81397](https://github.com/kubernetes/kubernetes/pull/81397), [@ddebroy](https://github.com/ddebroy)) 2006 2007 ## Dependencies 2008 2009 ### Changed 2010 2011 - the default Go version was updated to v1.12.9. ([#78958](https://github.com/kubernetes/kubernetes/pull/78958), [#79966](https://github.com/kubernetes/kubernetes/pull/79966), [#81390](https://github.com/kubernetes/kubernetes/pull/81390), [#81489](https://github.com/kubernetes/kubernetes/pull/81489)) 2012 - etcd has been updated to v3.3.15 ([#82199](https://github.com/kubernetes/kubernetes/pull/82199), [@dims](https://github.com/dims)) 2013 - CoreDNS for kubeadm and kube-up has been updated to v1.6.2 ([#82127](https://github.com/kubernetes/kubernetes/pull/82127)) 2014 - Cluster Autoscaler has been updated to v1.16.0 ([#82501](https://github.com/kubernetes/kubernetes/pull/82501), [@losipiuk](https://github.com/losipiuk)) 2015 - fluentd has been updated to v1.5.1 ([#79014](https://github.com/kubernetes/kubernetes/pull/79014)) 2016 - fluentd-elasticsearch plugin has been updated to v3.5.3 ([#79014](https://github.com/kubernetes/kubernetes/pull/79014)) 2017 - elasticsearch has been updated to v7.1.1 ([#79014](https://github.com/kubernetes/kubernetes/pull/79014)) 2018 - kibana has been updated to v7.1.1 ([#79014](https://github.com/kubernetes/kubernetes/pull/79014)) 2019 - Azure SDK and go-autorest API versions have been updated ([#79574](https://github.com/kubernetes/kubernetes/pull/79574)) 2020 - Azure API versions have been updated (container registry to 2018-09-01, network to 2018-08-01) ([#79583](https://github.com/kubernetes/kubernetes/pull/79583)) 2021 - kube-addon-manager has been updated to v9.0.2 ([#80861](https://github.com/kubernetes/kubernetes/pull/80861)) 2022 - golang/x/net has been updated to bring in fixes for CVE-2019-9512, CVE-2019-9514 ([#81394](https://github.com/kubernetes/kubernetes/pull/81394)) 2023 - GCE windows node image has been updated. ([#81106](https://github.com/kubernetes/kubernetes/pull/81106)) 2024 - portworx plugin has been updated on libopenstorage/openstorage to v1.0.0 ([#80495](https://github.com/kubernetes/kubernetes/pull/80495)) 2025 - metrics-server has been updated to v0.3.4 ([#82322](https://github.com/kubernetes/kubernetes/pull/82322), [@olagacek](https://github.com/olagacek)) 2026 - klog has been updated to v0.4.0 ([#81164](https://github.com/kubernetes/kubernetes/pull/81164)) 2027 2028 ### Unchanged 2029 2030 - The list of validated docker versions remains unchanged. 2031 - The current list is 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09. ([#72823](https://github.com/kubernetes/kubernetes/pull/72823), [#72831](https://github.com/kubernetes/kubernetes/pull/72831)) 2032 - CNI remains unchanged at v0.7.5. ([#75455](https://github.com/kubernetes/kubernetes/pull/75455)) 2033 - cri-tools remains unchanged at v1.14.0. ([#75658](https://github.com/kubernetes/kubernetes/pull/75658)) 2034 - CAdvisor remains unchanged at v0.33.2. ([#76291](https://github.com/kubernetes/kubernetes/pull/76291)) 2035 - event-exporter remains unchanged at v0.2.5. ([#77815](https://github.com/kubernetes/kubernetes/pull/77815)) 2036 - ip-masq-agent remains unchanged at v2.4.1. ([#77844](https://github.com/kubernetes/kubernetes/pull/77844)) 2037 - k8s-dns-node-cache remains unchanged at v1.15.1 ([#76640](https://github.com/kubernetes/kubernetes/pull/76640), [@george-angel](https://github.com/george-angel)) 2038 - CSI remains unchanged at to v1.1.0. ([#75391](https://github.com/kubernetes/kubernetes/pull/75391)) 2039 - The dashboard add-on remains unchanged at v1.10.1. ([#72495](https://github.com/kubernetes/kubernetes/pull/72495)) 2040 - kube-dns is unchanged at v1.14.13 as of Kubernetes 1.12. ([#68900](https://github.com/kubernetes/kubernetes/pull/68900)) 2041 - Influxdb is unchanged at v1.3.3 as of Kubernetes 1.10. ([#53319](https://github.com/kubernetes/kubernetes/pull/53319)) 2042 - Grafana is unchanged at v4.4.3 as of Kubernetes 1.10. ([#53319](https://github.com/kubernetes/kubernetes/pull/53319)) 2043 - The fluent-plugin-kubernetes_metadata_filter plugin in fluentd-elasticsearch is unchanged at v2.1.6. ([#71180](https://github.com/kubernetes/kubernetes/pull/71180)) 2044 - fluentd-gcp is unchanged at v3.2.0 as of Kubernetes 1.13. ([#70954](https://github.com/kubernetes/kubernetes/pull/70954)) 2045 - OIDC authentication is unchanged at coreos/go-oidc v2 as of Kubernetes 1.10. ([#58544](https://github.com/kubernetes/kubernetes/pull/58544)) 2046 - Calico is unchanged at v3.3.1 as of Kubernetes 1.13. ([#70932](https://github.com/kubernetes/kubernetes/pull/70932)) 2047 - GLBC remains unchanged at v1.2.3 as of Kubernetes 1.12. ([#66793](https://github.com/kubernetes/kubernetes/pull/66793)) 2048 - Ingress-gce remains unchanged at v1.2.3 as of Kubernetes 1.12. ([#66793](https://github.com/kubernetes/kubernetes/pull/66793)) 2049 2050 ### Removed 2051 2052 - Remove deprecated github.com/kardianos/osext dependency ([#80142](https://github.com/kubernetes/kubernetes/pull/80142)) 2053 2054 ### Detailed go Dependency Changes 2055 2056 #### Added 2057 2058 - github.com/Azure/go-autorest/autorest/adal: [v0.5.0](https://github.com/Azure/go-autorest/autorest/adal/tree/v0.5.0) 2059 - github.com/Azure/go-autorest/autorest/date: [v0.1.0](https://github.com/Azure/go-autorest/autorest/date/tree/v0.1.0) 2060 - github.com/Azure/go-autorest/autorest/mocks: [v0.2.0](https://github.com/Azure/go-autorest/autorest/mocks/tree/v0.2.0) 2061 - github.com/Azure/go-autorest/autorest/to: [v0.2.0](https://github.com/Azure/go-autorest/autorest/to/tree/v0.2.0) 2062 - github.com/Azure/go-autorest/autorest/validation: [v0.1.0](https://github.com/Azure/go-autorest/autorest/validation/tree/v0.1.0) 2063 - github.com/Azure/go-autorest/autorest: [v0.9.0](https://github.com/Azure/go-autorest/autorest/tree/v0.9.0) 2064 - github.com/Azure/go-autorest/logger: [v0.1.0](https://github.com/Azure/go-autorest/logger/tree/v0.1.0) 2065 - github.com/Azure/go-autorest/tracing: [v0.5.0](https://github.com/Azure/go-autorest/tracing/tree/v0.5.0) 2066 - github.com/armon/consul-api: [eb2c6b5](https://github.com/armon/consul-api/tree/eb2c6b5) 2067 - github.com/bifurcation/mint: [93c51c6](https://github.com/bifurcation/mint/tree/93c51c6) 2068 - github.com/caddyserver/caddy: [v1.0.3](https://github.com/caddyserver/caddy/tree/v1.0.3) 2069 - github.com/cenkalti/backoff: [v2.1.1+incompatible](https://github.com/cenkalti/backoff/tree/v2.1.1) 2070 - github.com/checkpoint-restore/go-criu: [bdb7599](https://github.com/checkpoint-restore/go-criu/tree/bdb7599) 2071 - github.com/cheekybits/genny: [9127e81](https://github.com/cheekybits/genny/tree/9127e81) 2072 - github.com/coredns/corefile-migration: [v1.0.2](https://github.com/coredns/corefile-migration/tree/v1.0.2) 2073 - github.com/coreos/go-etcd: [v2.0.0+incompatible](https://github.com/coreos/go-etcd/tree/v2.0.0) 2074 - github.com/dustin/go-humanize: [v1.0.0](https://github.com/dustin/go-humanize/tree/v1.0.0) 2075 - github.com/fatih/color: [v1.6.0](https://github.com/fatih/color/tree/v1.6.0) 2076 - github.com/flynn/go-shlex: [3f9db97](https://github.com/flynn/go-shlex/tree/3f9db97) 2077 - github.com/go-acme/lego: [v2.5.0+incompatible](https://github.com/go-acme/lego/tree/v2.5.0) 2078 - github.com/go-bindata/go-bindata: [v3.1.1+incompatible](https://github.com/go-bindata/go-bindata/tree/v3.1.1) 2079 - github.com/go-logr/logr: [v0.1.0](https://github.com/go-logr/logr/tree/v0.1.0) 2080 - github.com/google/martian: [v2.1.0+incompatible](https://github.com/google/martian/tree/v2.1.0) 2081 - github.com/google/pprof: [3ea8567](https://github.com/google/pprof/tree/3ea8567) 2082 - github.com/google/renameio: [v0.1.0](https://github.com/google/renameio/tree/v0.1.0) 2083 - github.com/googleapis/gax-go/v2: [v2.0.4](https://github.com/googleapis/gax-go/v2/tree/v2.0.4) 2084 - github.com/hashicorp/go-syslog: [v1.0.0](https://github.com/hashicorp/go-syslog/tree/v1.0.0) 2085 - github.com/jimstudt/http-authentication: [3eca13d](https://github.com/jimstudt/http-authentication/tree/3eca13d) 2086 - github.com/kisielk/errcheck: [v1.2.0](https://github.com/kisielk/errcheck/tree/v1.2.0) 2087 - github.com/kisielk/gotool: [v1.0.0](https://github.com/kisielk/gotool/tree/v1.0.0) 2088 - github.com/klauspost/cpuid: [v1.2.0](https://github.com/klauspost/cpuid/tree/v1.2.0) 2089 - github.com/kr/pty: [v1.1.5](https://github.com/kr/pty/tree/v1.1.5) 2090 - github.com/kylelemons/godebug: [d65d576](https://github.com/kylelemons/godebug/tree/d65d576) 2091 - github.com/lucas-clemente/aes12: [cd47fb3](https://github.com/lucas-clemente/aes12/tree/cd47fb3) 2092 - github.com/lucas-clemente/quic-clients: [v0.1.0](https://github.com/lucas-clemente/quic-clients/tree/v0.1.0) 2093 - github.com/lucas-clemente/quic-go-certificates: [d2f8652](https://github.com/lucas-clemente/quic-go-certificates/tree/d2f8652) 2094 - github.com/lucas-clemente/quic-go: [v0.10.2](https://github.com/lucas-clemente/quic-go/tree/v0.10.2) 2095 - github.com/marten-seemann/qtls: [v0.2.3](https://github.com/marten-seemann/qtls/tree/v0.2.3) 2096 - github.com/mattn/go-colorable: [v0.0.9](https://github.com/mattn/go-colorable/tree/v0.0.9) 2097 - github.com/mattn/go-isatty: [v0.0.3](https://github.com/mattn/go-isatty/tree/v0.0.3) 2098 - github.com/mholt/certmagic: [6a42ef9](https://github.com/mholt/certmagic/tree/6a42ef9) 2099 - github.com/mitchellh/go-homedir: [v1.1.0](https://github.com/mitchellh/go-homedir/tree/v1.1.0) 2100 - github.com/naoina/go-stringutil: [v0.1.0](https://github.com/naoina/go-stringutil/tree/v0.1.0) 2101 - github.com/naoina/toml: [v0.1.1](https://github.com/naoina/toml/tree/v0.1.1) 2102 - github.com/rogpeppe/go-internal: [v1.3.0](https://github.com/rogpeppe/go-internal/tree/v1.3.0) 2103 - github.com/thecodeteam/goscaleio: [v0.1.0](https://github.com/thecodeteam/goscaleio/tree/v0.1.0) 2104 - github.com/ugorji/go/codec: [d75b2dc](https://github.com/ugorji/go/codec/tree/d75b2dc) 2105 - github.com/xordataexchange/crypt: [b2862e3](https://github.com/xordataexchange/crypt/tree/b2862e3) 2106 - go.opencensus.io: v0.21.0 2107 - golang.org/x/mod: 4bf6d31 2108 - gopkg.in/airbrake/gobrake.v2: v2.0.9 2109 - gopkg.in/errgo.v2: v2.1.0 2110 - gopkg.in/gemnasium/logrus-airbrake-hook.v2: v2.1.2 2111 - gopkg.in/mcuadros/go-syslog.v2: v2.2.1 2112 - gotest.tools/gotestsum: v0.3.5 2113 - honnef.co/go/tools: v0.0.1-2019.2.2 2114 2115 #### Changed 2116 2117 - cloud.google.com/go: v0.34.0 → v0.38.0 2118 - github.com/Azure/azure-sdk-for-go: [v21.4.0+incompatible → v32.5.0+incompatible](https://github.com/Azure/azure-sdk-for-go/compare/v21.4.0...v32.5.0) 2119 - github.com/BurntSushi/toml: [v0.3.0 → v0.3.1](https://github.com/BurntSushi/toml/compare/v0.3.0...v0.3.1) 2120 - github.com/GoogleCloudPlatform/k8s-cloud-provider: [f8e9959 → 27a4ced](https://github.com/GoogleCloudPlatform/k8s-cloud-provider/compare/f8e9959...27a4ced) 2121 - github.com/PuerkitoBio/purell: [v1.1.0 → v1.1.1](https://github.com/PuerkitoBio/purell/compare/v1.1.0...v1.1.1) 2122 - github.com/asaskevich/govalidator: [f9ffefc → f61b66f](https://github.com/asaskevich/govalidator/compare/f9ffefc...f61b66f) 2123 - github.com/client9/misspell: [9ce5d97 → v0.3.4](https://github.com/client9/misspell/compare/9ce5d97...v0.3.4) 2124 - github.com/containernetworking/cni: [v0.6.0 → v0.7.1](https://github.com/containernetworking/cni/compare/v0.6.0...v0.7.1) 2125 - github.com/coreos/etcd: [v3.3.13+incompatible → v3.3.15+incompatible](https://github.com/coreos/etcd/compare/v3.3.13...v3.3.15) 2126 - github.com/coreos/go-oidc: [065b426 → v2.1.0+incompatible](https://github.com/coreos/go-oidc/compare/065b426...v2.1.0) 2127 - github.com/coreos/go-semver: [e214231 → v0.3.0](https://github.com/coreos/go-semver/compare/e214231...v0.3.0) 2128 - github.com/cpuguy83/go-md2man: [v1.0.4 → v1.0.10](https://github.com/cpuguy83/go-md2man/compare/v1.0.4...v1.0.10) 2129 - github.com/cyphar/filepath-securejoin: [ae69057 → v0.2.2](https://github.com/cyphar/filepath-securejoin/compare/ae69057...v0.2.2) 2130 - github.com/dgrijalva/jwt-go: [01aeca5 → v3.2.0+incompatible](https://github.com/dgrijalva/jwt-go/compare/01aeca5...v3.2.0) 2131 - github.com/docker/distribution: [edc3ab2 → v2.7.1+incompatible](https://github.com/docker/distribution/compare/edc3ab2...v2.7.1) 2132 - github.com/emicklei/go-restful: [ff4f55a → v2.9.5+incompatible](https://github.com/emicklei/go-restful/compare/ff4f55a...v2.9.5) 2133 - github.com/evanphx/json-patch: [5858425 → v4.2.0+incompatible](https://github.com/evanphx/json-patch/compare/5858425...v4.2.0) 2134 - github.com/fatih/camelcase: [f6a740d → v1.0.0](https://github.com/fatih/camelcase/compare/f6a740d...v1.0.0) 2135 - github.com/go-openapi/analysis: [v0.17.2 → v0.19.2](https://github.com/go-openapi/analysis/compare/v0.17.2...v0.19.2) 2136 - github.com/go-openapi/errors: [v0.17.2 → v0.19.2](https://github.com/go-openapi/errors/compare/v0.17.2...v0.19.2) 2137 - github.com/go-openapi/jsonpointer: [v0.19.0 → v0.19.2](https://github.com/go-openapi/jsonpointer/compare/v0.19.0...v0.19.2) 2138 - github.com/go-openapi/jsonreference: [v0.19.0 → v0.19.2](https://github.com/go-openapi/jsonreference/compare/v0.19.0...v0.19.2) 2139 - github.com/go-openapi/loads: [v0.17.2 → v0.19.2](https://github.com/go-openapi/loads/compare/v0.17.2...v0.19.2) 2140 - github.com/go-openapi/runtime: [v0.17.2 → v0.19.0](https://github.com/go-openapi/runtime/compare/v0.17.2...v0.19.0) 2141 - github.com/go-openapi/spec: [v0.17.2 → v0.19.2](https://github.com/go-openapi/spec/compare/v0.17.2...v0.19.2) 2142 - github.com/go-openapi/strfmt: [v0.17.0 → v0.19.0](https://github.com/go-openapi/strfmt/compare/v0.17.0...v0.19.0) 2143 - github.com/go-openapi/swag: [v0.17.2 → v0.19.2](https://github.com/go-openapi/swag/compare/v0.17.2...v0.19.2) 2144 - github.com/go-openapi/validate: [v0.18.0 → v0.19.2](https://github.com/go-openapi/validate/compare/v0.18.0...v0.19.2) 2145 - github.com/godbus/dbus: [c7fdd8b → v4.1.0+incompatible](https://github.com/godbus/dbus/compare/c7fdd8b...v4.1.0) 2146 - github.com/gogo/protobuf: [342cbe0 → 65acae2](https://github.com/gogo/protobuf/compare/342cbe0...65acae2) 2147 - github.com/golang/mock: [bd3c8e8 → v1.2.0](https://github.com/golang/mock/compare/bd3c8e8...v1.2.0) 2148 - github.com/golang/protobuf: [v1.2.0 → v1.3.1](https://github.com/golang/protobuf/compare/v1.2.0...v1.3.1) 2149 - github.com/google/btree: [7d79101 → 4030bb1](https://github.com/google/btree/compare/7d79101...4030bb1) 2150 - github.com/google/cadvisor: [9db8c7d → v0.34.0](https://github.com/google/cadvisor/compare/9db8c7d...v0.34.0) 2151 - github.com/google/gofuzz: [24818f7 → v1.0.0](https://github.com/google/gofuzz/compare/24818f7...v1.0.0) 2152 - github.com/google/uuid: [v1.0.0 → v1.1.1](https://github.com/google/uuid/compare/v1.0.0...v1.1.1) 2153 - github.com/gophercloud/gophercloud: [c818fa6 → v0.1.0](https://github.com/gophercloud/gophercloud/compare/c818fa6...v0.1.0) 2154 - github.com/gorilla/websocket: [4201258 → v1.4.0](https://github.com/gorilla/websocket/compare/4201258...v1.4.0) 2155 - github.com/grpc-ecosystem/go-grpc-prometheus: [2500245 → v1.2.0](https://github.com/grpc-ecosystem/go-grpc-prometheus/compare/2500245...v1.2.0) 2156 - github.com/hashicorp/golang-lru: [v0.5.0 → v0.5.1](https://github.com/hashicorp/golang-lru/compare/v0.5.0...v0.5.1) 2157 - github.com/hashicorp/hcl: [d8c773c → v1.0.0](https://github.com/hashicorp/hcl/compare/d8c773c...v1.0.0) 2158 - github.com/heketi/heketi: [558b292 → v9.0.0+incompatible](https://github.com/heketi/heketi/compare/558b292...v9.0.0) 2159 - github.com/jonboulle/clockwork: [72f9bd7 → v0.1.0](https://github.com/jonboulle/clockwork/compare/72f9bd7...v0.1.0) 2160 - github.com/json-iterator/go: [ab8a2e0 → v1.1.7](https://github.com/json-iterator/go/compare/ab8a2e0...v1.1.7) 2161 - github.com/kr/pretty: [f31442d → v0.1.0](https://github.com/kr/pretty/compare/f31442d...v0.1.0) 2162 - github.com/kr/text: [6807e77 → v0.1.0](https://github.com/kr/text/compare/6807e77...v0.1.0) 2163 - github.com/libopenstorage/openstorage: [093a0c3 → v1.0.0](https://github.com/libopenstorage/openstorage/compare/093a0c3...v1.0.0) 2164 - github.com/magiconair/properties: [61b492c → v1.8.1](https://github.com/magiconair/properties/compare/61b492c...v1.8.1) 2165 - github.com/mailru/easyjson: [60711f1 → 94de47d](https://github.com/mailru/easyjson/compare/60711f1...94de47d) 2166 - github.com/mattn/go-shellwords: [f8471b0 → v1.0.5](https://github.com/mattn/go-shellwords/compare/f8471b0...v1.0.5) 2167 - github.com/miekg/dns: [5d001d0 → v1.1.4](https://github.com/miekg/dns/compare/5d001d0...v1.1.4) 2168 - github.com/mistifyio/go-zfs: [1b4ae6f → v2.1.1+incompatible](https://github.com/mistifyio/go-zfs/compare/1b4ae6f...v2.1.1) 2169 - github.com/mitchellh/go-wordwrap: [ad45545 → v1.0.0](https://github.com/mitchellh/go-wordwrap/compare/ad45545...v1.0.0) 2170 - github.com/mvdan/xurls: [1b768d7 → v1.1.0](https://github.com/mvdan/xurls/compare/1b768d7...v1.1.0) 2171 - github.com/onsi/ginkgo: [v1.6.0 → v1.8.0](https://github.com/onsi/ginkgo/compare/v1.6.0...v1.8.0) 2172 - github.com/onsi/gomega: [5533ce8 → v1.5.0](https://github.com/onsi/gomega/compare/5533ce8...v1.5.0) 2173 - github.com/opencontainers/go-digest: [a6d0ee4 → v1.0.0-rc1](https://github.com/opencontainers/go-digest/compare/a6d0ee4...v1.0.0-rc1) 2174 - github.com/opencontainers/image-spec: [372ad78 → v1.0.1](https://github.com/opencontainers/image-spec/compare/372ad78...v1.0.1) 2175 - github.com/opencontainers/runc: [f000fe1 → 6cc5158](https://github.com/opencontainers/runc/compare/f000fe1...6cc5158) 2176 - github.com/opencontainers/selinux: [4a2974b → v1.2.2](https://github.com/opencontainers/selinux/compare/4a2974b...v1.2.2) 2177 - github.com/robfig/cron: [df38d32 → v1.1.0](https://github.com/robfig/cron/compare/df38d32...v1.1.0) 2178 - github.com/russross/blackfriday: [300106c → v1.5.2](https://github.com/russross/blackfriday/compare/300106c...v1.5.2) 2179 - github.com/seccomp/libseccomp-golang: [1b506fc → v0.9.1](https://github.com/seccomp/libseccomp-golang/compare/1b506fc...v0.9.1) 2180 - github.com/sirupsen/logrus: [v1.2.0 → v1.4.2](https://github.com/sirupsen/logrus/compare/v1.2.0...v1.4.2) 2181 - github.com/spf13/afero: [b28a7ef → v1.2.2](https://github.com/spf13/afero/compare/b28a7ef...v1.2.2) 2182 - github.com/spf13/cast: [e31f36f → v1.3.0](https://github.com/spf13/cast/compare/e31f36f...v1.3.0) 2183 - github.com/spf13/cobra: [c439c4f → v0.0.5](https://github.com/spf13/cobra/compare/c439c4f...v0.0.5) 2184 - github.com/spf13/jwalterweatherman: [33c24e7 → v1.1.0](https://github.com/spf13/jwalterweatherman/compare/33c24e7...v1.1.0) 2185 - github.com/spf13/pflag: [v1.0.1 → v1.0.3](https://github.com/spf13/pflag/compare/v1.0.1...v1.0.3) 2186 - github.com/spf13/viper: [7fb2782 → v1.3.2](https://github.com/spf13/viper/compare/7fb2782...v1.3.2) 2187 - github.com/stretchr/objx: [v0.1.1 → v0.2.0](https://github.com/stretchr/objx/compare/v0.1.1...v0.2.0) 2188 - github.com/stretchr/testify: [v1.2.2 → v1.3.0](https://github.com/stretchr/testify/compare/v1.2.2...v1.3.0) 2189 - golang.org/x/net: 65e2d4e → cdfb69a 2190 - golang.org/x/tools: aa82965 → 6e04913 2191 - google.golang.org/api: 583d854 → 5213b80 2192 - google.golang.org/genproto: 09f6ed2 → 54afdca 2193 - google.golang.org/grpc: v1.13.0 → v1.23.0 2194 - gopkg.in/check.v1: 20d25e2 → 788fd78 2195 - gopkg.in/natefinch/lumberjack.v2: 20b71e5 → v2.0.0 2196 - gopkg.in/square/go-jose.v2: 89060de → v2.2.2 2197 - gopkg.in/yaml.v2: v2.2.1 → v2.2.2 2198 - k8s.io/gengo: f8a0810 → 26a6646 2199 - k8s.io/klog: v0.3.1 → v0.4.0 2200 - k8s.io/kube-openapi: b3a7cee → 743ec37 2201 - k8s.io/utils: c2654d5 → 581e001 2202 - sigs.k8s.io/structured-merge-diff: e85c7b2 → 6149e45 2203 2204 #### Removed 2205 2206 - github.com/Azure/go-autorest: [v11.1.2+incompatible](https://github.com/Azure/go-autorest/tree/v11.1.2) 2207 - github.com/codedellemc/goscaleio: [20e2ce2](https://github.com/codedellemc/goscaleio/tree/20e2ce2) 2208 - github.com/d2g/dhcp4: [a1d1b6c](https://github.com/d2g/dhcp4/tree/a1d1b6c) 2209 - github.com/d2g/dhcp4client: [6e570ed](https://github.com/d2g/dhcp4client/tree/6e570ed) 2210 - github.com/jteeuwen/go-bindata: [a0ff256](https://github.com/jteeuwen/go-bindata/tree/a0ff256) 2211 - github.com/kardianos/osext: [8fef92e](https://github.com/kardianos/osext/tree/8fef92e) 2212 - github.com/kr/fs: [2788f0d](https://github.com/kr/fs/tree/2788f0d) 2213 - github.com/marstr/guid: [8bdf7d1](https://github.com/marstr/guid/tree/8bdf7d1) 2214 - github.com/mholt/caddy: [2de4950](https://github.com/mholt/caddy/tree/2de4950) 2215 - github.com/natefinch/lumberjack: [v2.0.0+incompatible](https://github.com/natefinch/lumberjack/tree/v2.0.0) 2216 - github.com/pkg/sftp: [4d0e916](https://github.com/pkg/sftp/tree/4d0e916) 2217 - github.com/shurcooL/sanitized_anchor_name: [10ef21a](https://github.com/shurcooL/sanitized_anchor_name/tree/10ef21a) 2218 - github.com/sigma/go-inotify: [c87b6cf](https://github.com/sigma/go-inotify/tree/c87b6cf) 2219 - github.com/vmware/photon-controller-go-sdk: [4a435da](https://github.com/vmware/photon-controller-go-sdk/tree/4a435da) 2220 - github.com/xanzy/go-cloudstack: [1e2cbf6](https://github.com/xanzy/go-cloudstack/tree/1e2cbf6) 2221 - gopkg.in/yaml.v1: 9f9df34 2222 - [v1.16.0-rc.2](#v1160-rc2) 2223 - [v1.16.0-rc.1](#v1160-rc1) 2224 - [v1.16.0-beta.2](#v1160-beta2) 2225 - [v1.16.0-beta.1](#v1160-beta1) 2226 - [v1.16.0-alpha.3](#v1160-alpha3) 2227 - [v1.16.0-alpha.2](#v1160-alpha2) 2228 - [v1.16.0-alpha.1](#v1160-alpha1) 2229 2230 2231 2232 # v1.16.0-rc.2 2233 2234 [Documentation](https://docs.k8s.io) 2235 2236 ## Downloads for v1.16.0-rc.2 2237 2238 2239 filename | sha512 hash 2240 -------- | ----------- 2241 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes.tar.gz) | `68837f83bcf380e22b50f145fb64404584e96e5714a6c0cbc1ba76e290dc267f6b53194e2b51f19c1145ae7c3e5874124d35ff430cda15f67b0f9c954803389c` 2242 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-src.tar.gz) | `922552ed60d425fa6d126ffb34db6a7f123e1b9104e751edaed57b4992826620383446e6cf4f8a9fd55aac72f95a69b45e53274a41aaa838c2c2ae15ff4ddad2` 2243 2244 ### Client Binaries 2245 2246 filename | sha512 hash 2247 -------- | ----------- 2248 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-darwin-386.tar.gz) | `d0df8f57f4d9c2822badc507345f82f87d0e8e49c79ca907a0e4e4dd634db964b84572f88b8ae7eaf50a20965378d464e0d1e7f588e84e926edfb741b859e7d2` 2249 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-darwin-amd64.tar.gz) | `0bc7daaf1165189b57dcdbe59f402731830b6f4db53b853350056822602579d52fe43ce5ac6b7d4b6d89d81036ae94eab6b7167e78011a96792acfbf6892fa39` 2250 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-linux-386.tar.gz) | `7735c607bb99b47924140a6a3e794912b2b97b6b54024af1de5db6765b8cc518cba6b145c25dc67c8d8f827805d9a61f676b4ae67b8ef86cfda2fe76de822c6a` 2251 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-linux-amd64.tar.gz) | `d35f70cea4780a80c24588bc760c38c138d73e5f80f9fe89d952075c24cbf179dd504c2bd7ddb1756c2632ffbcc69a334684710a2d702443043998f66bec4a25` 2252 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-linux-arm.tar.gz) | `e1fc50b6884c42e92649a231db60e35d4e13e58728e4af7f6eca8b0baa719108cdd960db1f1dbd623085610dbccf7f17df733de1faf10ebf6cd1977ecd7f6213` 2253 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-linux-arm64.tar.gz) | `defc25fe403c20ef322b2149be28a5b44c28c7284f11bcf193a07d7f45110ce2bd6227d3a4aa48859aaeb67796809962785651ca9f76121fb9534366b40c4b7d` 2254 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-linux-ppc64le.tar.gz) | `e87b16c948d09ddbc5d6e3fab05ad3c5a58aa7836d4f42c59edab640465531869c92ecdfa2845ec3eecd95b8ccba3dafdd9337f4c313763c6e5105b8740f2dca` 2255 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-linux-s390x.tar.gz) | `2c25a1860fa81cea05a1840d6a200a3a794cc50cfe45a4efec57d7122208b1354e86f698437bbe5c915d6fb70ef9525f844edc0fa63387ab8c1586a6b22008a5` 2256 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-windows-386.tar.gz) | `267654a7ecfa37c800c1c94ea78343f5466783881cfac62091cfbd8c62489f04bd74a7a39a08253cb51d7ba52c207f56da371f992f61c1468b595c094f0e080f` 2257 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-client-windows-amd64.tar.gz) | `bd4c25b80e54f9fc0c07f64550d020878f899e4e3a28ca57dd532fdbab9ab700d296d2890185591ac27bce6fde336ab90f3102a6797e174d233db76f24f5ac1b` 2258 2259 ### Server Binaries 2260 2261 filename | sha512 hash 2262 -------- | ----------- 2263 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-server-linux-amd64.tar.gz) | `13a93bb9bd5599b669af7bd25537ee81cefd6d8c73bedfbac845703c01950c70b2aa39f94f2346d935bc167bae435dbcd6e1758341b634102265657e1b1c1259` 2264 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-server-linux-arm.tar.gz) | `781d127f32d8479bc21beed855ec73e383702e6e982854138adce8edb0ee4d1d4b0c6e723532bc761689d17512c18b1945d05b0e4adb3fe4b98428cce40d52c8` 2265 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-server-linux-arm64.tar.gz) | `6d6dfa49288e4a4ce77ca4f7e83a51c78a2b1844dd95df10cb12fff5a104e750d8e4e117b631448e066487c4c71648e822c87ed83a213f17f27f8c7ecb328ca4` 2266 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-server-linux-ppc64le.tar.gz) | `97804d87ea984167fdbdedcfb38380bd98bb2ef150c1a631c6822905ce5270931a907226d5ddefc8d98d5326610daa79a08964fc4d7e8b438832beb966efd214` 2267 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-server-linux-s390x.tar.gz) | `d45bd651c7f4b6e62ceb661c2ec70afca06a8d1fde1e50bb7783d05401c37823cf21b9f0d3ac87e6b91eeec9d03fc539c3713fd46beff6207e8ebac1bf9d1dd5` 2268 2269 ### Node Binaries 2270 2271 filename | sha512 hash 2272 -------- | ----------- 2273 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-node-linux-amd64.tar.gz) | `42c57b59ce43f8961e427d622ee9cfa85cc23468779945262d59aa8cd31afd495c7abaaef7263b9db60ec939ba5e9898ebc3281e8ec81298237123ce4739cbff` 2274 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-node-linux-arm.tar.gz) | `034a5611909df462ef6408f5ba5ff5ebfb4e1178b2ad06a59097560040c4fcdb163faec48ab4297ca6c21282d7b146f9a5eebd3f2573f7d6d7189d6d29f2cf34` 2275 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-node-linux-arm64.tar.gz) | `df1493fa2d67b59eaf02096889223bbf0d71797652d3cbd89e8a3106ff6012ea17d25daaa4baf9f26c2e061afb4b69e3e6814ba66e9c4744f04230c922fbc251` 2276 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-node-linux-ppc64le.tar.gz) | `812a5057bbf832c93f741cc39d04fc0087e36b81b6b123ec5ef02465f7ab145c5152cfc1f7c76032240695c7d7ab71ddb9a2a4f5e1f1a2abb63f32afa3fb6c7c` 2277 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-node-linux-s390x.tar.gz) | `2a58a4b201631789d4309ddc665829aedcc05ec4fe6ad6e4d965ef3283a381b8a4980b4b728cfe9a38368dac49921f61ac6938f0208b671afd2327f2013db22a` 2278 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.2/kubernetes-node-windows-amd64.tar.gz) | `7fb09e7667715f539766398fc1bbbc4bf17c64913ca09d4e3535dfc4d1ba2bf6f1a3fcc6d81dbf473ba3f10fd29c537ce5debc17268698048ce7b378802a6c46` 2279 2280 ## Changelog since v1.16.0-rc.1 2281 2282 ### Other notable changes 2283 2284 * Single static pod files and pod files from http endpoints cannot be larger than 10 MB. HTTP probe payloads are now truncated to 10KB. ([#82669](https://github.com/kubernetes/kubernetes/pull/82669), [@rphillips](https://github.com/rphillips)) 2285 * Restores compatibility with <=1.15.x custom resources by not publishing OpenAPI for non-structural custom resource definitions ([#82653](https://github.com/kubernetes/kubernetes/pull/82653), [@liggitt](https://github.com/liggitt)) 2286 * Fixes regression in logging spurious stack traces when proxied connections are closed by the backend ([#82588](https://github.com/kubernetes/kubernetes/pull/82588), [@liggitt](https://github.com/liggitt)) 2287 2288 2289 2290 # v1.16.0-rc.1 2291 2292 [Documentation](https://docs.k8s.io) 2293 2294 ## Downloads for v1.16.0-rc.1 2295 2296 2297 filename | sha512 hash 2298 -------- | ----------- 2299 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes.tar.gz) | `2feadb470a8b0d498dff2c122d792109bc48e24bfc7f49b4b2b40a268061c83d9541cbcf902f2b992d6675e38d69ccdded9435ac488e041ff73d0c2dc518a5a9` 2300 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-src.tar.gz) | `6d8877e735e041c989c0fca9dd9e57e5960299e74f66f69907b5e1265419c69ed3006c0161e0ced63073e28073355a5627154cf5db53b296b4a209b006b45db0` 2301 2302 ### Client Binaries 2303 2304 filename | sha512 hash 2305 -------- | ----------- 2306 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-darwin-386.tar.gz) | `27bbfcb709854a9625dbb22c357492c1818bc1986b94e8cf727c046d596c4f1fe385df5b2ce61baaf95b066d584a8c04190215eaf979e12707c6449766e84810` 2307 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-darwin-amd64.tar.gz) | `9c2ea22e188282356cd532801cb94d799bde5a5716f037b81e7f83273f699bf80776b253830e3a4e1a72c420f0c0b84e28ae043c9d28a49e9455e6b1449a353c` 2308 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-linux-386.tar.gz) | `bbba78b8f972d0c247ed11e88010fc934a694efce8d2605635902b4a22f5ecc7e710f640bcefbba97ef28f6db68b9d8fb9e6a4a099603493c1ddcc5fd50c0d17` 2309 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-linux-amd64.tar.gz) | `f2f04dc9b93d1c8f5295d3f559a3abdd19ea7011741aa006b2cd96542c06a6892d7ed2bad8479c89e7d6ae0ed0685e68d5096bd5a46431c8cab8a90c04f1f00c` 2310 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-linux-arm.tar.gz) | `77d1f5b4783f7480d879d0b7682b1d46e730e7fb8edbc6eccd96986c31ceecbf123cd9fd11c5a388218a8c693b1b545daed28ca88f36ddaca06adac4422e4be5` 2311 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-linux-arm64.tar.gz) | `0b57aa1dbbce51136789cb373d93e641d1f095a4bc9695d60917e85c814c8959a4d6e33224dc86295210d01e73e496091a191f303348f3b652a2b6160b1e6059` 2312 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-linux-ppc64le.tar.gz) | `847065d541dece0fc931947146dbc90b181f923137772f26c7c93476e022f4f654e00f9928df7a13a9dec27075dd8134bdb168b5c57d4efa29ed20a6a2112272` 2313 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-linux-s390x.tar.gz) | `d7e8a808da9e2551ca7d8e7cb25222cb9ac01595f78ebbc86152ae1c21620d4d8478ef3d374d69f47403ca913fc716fbaa81bd3ff082db2fc5814ef8dc66eeec` 2314 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-windows-386.tar.gz) | `c9cf6a6b9f2f29152af974d30f3fd97ba33693d5cbbf8fc76bcf6590979e7ac8307e5da4f84a646cec6b68f6fa1a83aa1ce24eb6429baa0a39c92d5901bd80be` 2315 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-client-windows-amd64.tar.gz) | `ebea0c0b64d251e6023e8a5a100aa609bc278c797170765da2e35c8997efc233bec9f8d1436aeee1cd6459e30ec78ba64b84de47c26a4e4645e153e5e598202b` 2316 2317 ### Server Binaries 2318 2319 filename | sha512 hash 2320 -------- | ----------- 2321 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-server-linux-amd64.tar.gz) | `2fe7ccce15e705826c4ccfce48df8130ba89a0c930bca4b61f49267e9d490f57cf6220671752e44e55502bee501a9af2f0ac3927378a87b466f2526fa6e45834` 2322 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-server-linux-arm.tar.gz) | `6eb77e59095a1de9eb21e7065e8d10b7d0baf1888991a42089ede6d4f8a8cac0b17ae793914eef5796d56d8f0b958203d5df1f7ed45856dce7244c9f047f9793` 2323 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-server-linux-arm64.tar.gz) | `429ce0d5459384c9d3a2bb103924eebc4c30343c821252dde8f4413fcf29cc73728d378bfd193c443479bde6bfd26e0a13c036d4d4ae22034d66f6cad70f684d` 2324 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-server-linux-ppc64le.tar.gz) | `18041d9c99efc00c8c9dbb6444974efdbf4969a4f75faea75a3c859b1ee8485d2bf3f01b7942a524dcd6a71c82af7a5937fc9120286e920cf2d501b7c76ab160` 2325 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-server-linux-s390x.tar.gz) | `2124c3d8856e50ca6b2b61d83f108ab921a1217fac2a80bf765d51b68f4e67d504471787d375524974173782aa37c57b6bf1fc6c7704ed7e6cabe15ec3c543b1` 2326 2327 ### Node Binaries 2328 2329 filename | sha512 hash 2330 -------- | ----------- 2331 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-node-linux-amd64.tar.gz) | `ea1bcd8cc51fbc95058a8a592eb454c07ab5dadc1c237bbc59f278f8adc46bda1f334e73463e1edbd6da5469c4a527ceb1cb0a96686493d3ff4e8878dd1c9a20` 2332 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-node-linux-arm.tar.gz) | `e5d62df5fd086ff5712f59f71ade9efcf617a13c567de965ce54c79f3909372bed4edbf6639cf058fe1d5c4042f794e1c6a91e5e20d9dcce597a95dedf2474b2` 2333 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-node-linux-arm64.tar.gz) | `5aa0a7a3d02b65253e4e814e51cea6dd895170f2838fea02f94e4efd3f938dbf83bc7f209801856b98420373c04147fab9cb8791d24d51dcedf960068dfe6fda` 2334 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-node-linux-ppc64le.tar.gz) | `f54bc5ae188f8ecb3ddcae20e06237430dd696f444a5c65b0aa3be79ad85c5b500625fa47ed0e126f6e738eb5d9ee082b52482a6913ec6d22473520fa6582e66` 2335 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-node-linux-s390x.tar.gz) | `afa4f9b747fff20ed03d40092a2df60dbd6ced0de7fd0c83c001866c4fe5b7117468e2f8c73cbef26f376b69b4750f188143076953fc200e8a5cc002c8ac705b` 2336 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-rc.1/kubernetes-node-windows-amd64.tar.gz) | `e9b76014a1d4268ad66ade06883dd3344c6312ece14ee988af645bdf9c5e9b62c31a0e339f774c67799b777314db6016d86a3753855c7d2eb461fbbf4e154ae7` 2337 2338 ## Changelog since v1.16.0-beta.2 2339 2340 ### Other notable changes 2341 2342 * Update Cluster Autoscaler to 1.16.0; changelog: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.16.0 ([#82501](https://github.com/kubernetes/kubernetes/pull/82501), [@losipiuk](https://github.com/losipiuk)) 2343 * Resolved regression serving custom resources with unhandled validation schemas with the ServerSideApply feature enabled ([#82438](https://github.com/kubernetes/kubernetes/pull/82438), [@liggitt](https://github.com/liggitt)) 2344 * Fix filter plugins are not been called during preemption ([#81876](https://github.com/kubernetes/kubernetes/pull/81876), [@wgliang](https://github.com/wgliang)) 2345 * Fix the dns suffix search list for GCE window clusters. ([#82314](https://github.com/kubernetes/kubernetes/pull/82314), [@lzang](https://github.com/lzang)) 2346 * Install and start logging agent based on kube env ENABLE_NODE_LOGGING. ([#81300](https://github.com/kubernetes/kubernetes/pull/81300), [@liyanhui1228](https://github.com/liyanhui1228)) 2347 * kubeadm: Allow users to skip the kube-proxy init addon phase during init and still be able to join a cluster and perform some other minor operations (but not upgrade). ([#82248](https://github.com/kubernetes/kubernetes/pull/82248), [@rosti](https://github.com/rosti)) 2348 * Bump metrics-server to v0.3.4 ([#82322](https://github.com/kubernetes/kubernetes/pull/82322), [@olagacek](https://github.com/olagacek)) 2349 * Updated default etcd service used by kubernetes to 3.3.15 ([#82199](https://github.com/kubernetes/kubernetes/pull/82199), [@dims](https://github.com/dims)) 2350 2351 2352 2353 # v1.16.0-beta.2 2354 2355 [Documentation](https://docs.k8s.io) 2356 2357 ## Downloads for v1.16.0-beta.2 2358 2359 2360 filename | sha512 hash 2361 -------- | ----------- 2362 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes.tar.gz) | `d1f4e9badc6a4422b9a261a5375769d63f0cac7fff2aff4122a325417b77d5e5317ba76a180cda2baa9fb1079c33e396fc16f82b31eeebea61004b0aabdf8c32` 2363 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-src.tar.gz) | `2ab20b777311746bf9af0947a2bea8ae36e27da7d917149518d7c2d2612f513bbf88d1f2c7efff6dc169aa43c2dd3be73985ef619172d50d99faa56492b35ce4` 2364 2365 ### Client Binaries 2366 2367 filename | sha512 hash 2368 -------- | ----------- 2369 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-darwin-386.tar.gz) | `55523fd5cfce0c5b79e981c6a4d5572790cfe4488ed23588be45ee13367e374cf703f611769751583986557b2607f271704d9f27e03f558e35e7c75796476b10` 2370 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-darwin-amd64.tar.gz) | `13e696782713da96f5fb2c3fa54d99ca40bc71262cb2cbc8e77a6d19ffd33b0767d3f27e693aa84103aca465f9b00ed109996d3579b4bd28566b8998212a0872` 2371 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-linux-386.tar.gz) | `7f4818599b84712edd2bf1d94f02f9a53c1f827b428a888356e793ff62e897276afcbc97f03bc0317e7d729740410037c57e6443f65c691eb959b676833511fa` 2372 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-linux-amd64.tar.gz) | `8a2656289d7d86cbded42831f6bc660b579609622c16428cf6cc782ac8b52df4c8511c5aad65aa520f398a65e35dee6ea5b5ad8e5fd14c5a8690a7248dc4c109` 2373 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-linux-arm.tar.gz) | `418606bc109b9acb2687ed297fa2eec272e8cb4ad3ce1173acd15a4b43cec0ecfd95e944faeecf862b349114081dd99dfac8615dc95cffc1cd4983c5b38e9c4e` 2374 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-linux-arm64.tar.gz) | `2eb943b745c270cd161e01a12195cfb38565de892a1da89e851495fb6f9d6664055e384e30d3551c25f120964e816e44df5415aff7c12a8639c30a42271abef7` 2375 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-linux-ppc64le.tar.gz) | `262e7d61e167e7accd43c47e9ce28323ae4614939a5af09ecc1023299cd2580220646e7c90d31fee0a17302f5d9df1e7da1e6774cc7e087248666b33399e8821` 2376 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-linux-s390x.tar.gz) | `8f0cfe669a211423dd697fdab722011ea9641ce3db64debafa539d4a424dd26065c8de5da7502a4d40235ff39158f3935bd337b807a63771391dffb282563ccf` 2377 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-windows-386.tar.gz) | `b1deab89653f4cd3ad8ad68b8ec3e1c038d1ef35bd2e4475d71d4781acf0b2002443f9c2b7d2cf06cbb9c568bea3881c06d723b0529cc8210f99450dc2dc5e43` 2378 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-client-windows-amd64.tar.gz) | `0e3b5150767efd0ed5d60b2327d2b7f6f2bda1a3532fca8e84a7ca161f6e069fae15af37d3fe8a641d34c9a65fc61f1c44dd3265ef6cacfd2df55c9c004bc6bd` 2379 2380 ### Server Binaries 2381 2382 filename | sha512 hash 2383 -------- | ----------- 2384 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-server-linux-amd64.tar.gz) | `32688295df1fcdb9472ed040dc5e8b19d04d62789d2eca64cfe08080d08ffee1eaa4853ce40bd336aabd2f764dd65b36237d4f9f1c697e2d6572861c0c8eff01` 2385 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-server-linux-arm.tar.gz) | `c8ea6d66e966889a54194f9dce2021131e9bae34040c56d8839341c47fc4074d6322cc8aadce28e7cdcee88ec79d37a73d52276deb1cc1eee231e4d3083d54e5` 2386 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-server-linux-arm64.tar.gz) | `12b42cfa33ff824392b81a604b7edcab95ecc67cddfc24c47ef67adb356a333998bc7b913b00daf7a213692d8d441153904474947b46c7f76ef03d4b2a63eab0` 2387 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-server-linux-ppc64le.tar.gz) | `e03f0eba181c03ddb7535e56ff330dafebb7dcb40889fd04f5609617ebb717f9f833e89810bff36d5299f72ae75d356fffb80f7b3bab2232c7597abcc003b8ba` 2388 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-server-linux-s390x.tar.gz) | `4e7bd061317a3445ad4b6b308f26218777677a1fef5fda181ee1a19e532a758f6bd3746a3fe1917a057ed71c94892aeaf00dd4eb008f61418ec3c80169a1f057` 2389 2390 ### Node Binaries 2391 2392 filename | sha512 hash 2393 -------- | ----------- 2394 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-node-linux-amd64.tar.gz) | `dc5606c17f0191afc6f28dce5ab566fd8f21a69fa3989a1c8f0976d7b8ccd32e26bb21e9fec9f4529c5a6c8301747d278484688a0592da291866f8fa4893dcbb` 2395 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-node-linux-arm.tar.gz) | `3d5d9893e06fd7be51dca11182ecb9e93108e86af40298fe66bb62e5e86f0bf4713667ba63d00b02cfddaf20878dd78cc738e76bf1ca715bbbe79347ca518ec4` 2396 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-node-linux-arm64.tar.gz) | `fd18a02f32aeafc5cce8f3f2eadd0e532857bd5264b7299b4e48f458f77ebaa53be94b1d1fe2062168f9d88c8a97e6c2d904fc3401a2d9e69dd4e8c87d01d915` 2397 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-node-linux-ppc64le.tar.gz) | `703afd80140db2fae897d83b3d2bc8889ff6c6249bb79be7a1cce6f0c9326148d22585a5249c2e976c69a2518e3f887eef4c9dc4a970ebb854a78e72c1385ccb` 2398 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-node-linux-s390x.tar.gz) | `445d4ef4f9d63eabe3b7c16114906bc450cfde3e7bf7c8aedd084c79a5e399bd24a7a9c2283b58d382fb11885bb2b412773a36fffb6fc2fac15d696439a0b800` 2399 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.2/kubernetes-node-windows-amd64.tar.gz) | `88b04171c3c0134044b7555fbc9b88071f5a73dbf2dac21f8a27b394b0870dff349a56b0ee4d8e1d9cfbeb98645e485f40b8d8863f3f3e833cba0ca6b1383ccf` 2400 2401 ## Changelog since v1.16.0-beta.1 2402 2403 ### Other notable changes 2404 2405 * Fix a bug in apiserver that could cause a valid update request to be rejected with a precondition check failure. ([#82303](https://github.com/kubernetes/kubernetes/pull/82303), [@roycaihw](https://github.com/roycaihw)) 2406 * Webhook client credentials configured with `--admission-control-config-file` must include non-default ports in the configured hostnames. For example, a webhook configured to speak to port 8443 on service `mysvc` in namespace `myns` would specify client credentials in a stanza with `name: mysvc.myns.svc:8443`. See https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#authenticate-apiservers for more details. ([#82252](https://github.com/kubernetes/kubernetes/pull/82252), [@liggitt](https://github.com/liggitt)) 2407 * Ensure the KUBE-MARK-DROP chain in kube-proxy mode=ipvs. The chain is ensured for both ipv4 and ipv6 in dual-stack operation. ([#82214](https://github.com/kubernetes/kubernetes/pull/82214), [@uablrek](https://github.com/uablrek)) 2408 * `kubectl cp` no longer supports copying symbolic links from containers; to support this use case, see `kubectl exec --help` for examples using `tar` directly ([#82143](https://github.com/kubernetes/kubernetes/pull/82143), [@soltysh](https://github.com/soltysh)) 2409 * kubeadm: fix for HTTPProxy check for IPv6 addresses (kubernetes/kubeadm#1769) ([#82267](https://github.com/kubernetes/kubernetes/pull/82267), [@kad](https://github.com/kad)) 2410 * Add PodOverhead awareness to kubectl ([#81929](https://github.com/kubernetes/kubernetes/pull/81929), [@egernst](https://github.com/egernst)) 2411 * The nbf (not before) claim, if present in ID token, is now enforced. ([#81413](https://github.com/kubernetes/kubernetes/pull/81413), [@anderseknert](https://github.com/anderseknert)) 2412 * Server-side apply will now use the openapi provided in the CRD validation field to help figure out how to correctly merge objects and update ownership. ([#77354](https://github.com/kubernetes/kubernetes/pull/77354), [@jennybuckley](https://github.com/jennybuckley)) 2413 * - Fix disk stats in LXD using ZFS storage pool ([#81972](https://github.com/kubernetes/kubernetes/pull/81972), [@dashpole](https://github.com/dashpole)) 2414 * - Fix CRI-O missing network metris bug 2415 * - Add `container_sockets`, `container_threads`, and `container_threads_max` metrics 2416 * Adds Endpoint Slice support for kubectl when discovery API group is enabled. ([#81795](https://github.com/kubernetes/kubernetes/pull/81795), [@robscott](https://github.com/robscott)) 2417 * Node-Problem-Detector v0.7.1 is used for addon daemonset. ([#82140](https://github.com/kubernetes/kubernetes/pull/82140), [@wangzhen127](https://github.com/wangzhen127)) 2418 * aggregated discovery requests can now timeout. Aggregated apiserver *must* complete discovery calls within five seconds. Other requests can take longer. ([#82146](https://github.com/kubernetes/kubernetes/pull/82146), [@deads2k](https://github.com/deads2k)) 2419 * Use feature gate `EnableAggregatedDiscoveryTimeout=false` if you *must* remove this check, but that feature gate will be removed next release. 2420 * Graduating Windows GMSA support from alpha to beta ([#82110](https://github.com/kubernetes/kubernetes/pull/82110), [@wk8](https://github.com/wk8)) 2421 * Add UnschedulableAndUnresolvable status code for Scheduler Framework ([#82034](https://github.com/kubernetes/kubernetes/pull/82034), [@alculquicondor](https://github.com/alculquicondor)) 2422 * Kubeadm now includes CoreDNS version 1.6.2 ([#82127](https://github.com/kubernetes/kubernetes/pull/82127), [@rajansandeep](https://github.com/rajansandeep)) 2423 * - The CoreDNS Deployment now checks readiness via the `ready` plugin. 2424 * - The `proxy` plugin has been deprecated. The `forward` plugin is to be used instead. 2425 * - `kubernetes` plugin removes the `resyncperiod` option. 2426 * - The `upstream` option is deprecated and ignored if included. 2427 * Make kubectl get --ignore-not-found continue processing when encountering error. ([#82120](https://github.com/kubernetes/kubernetes/pull/82120), [@soltysh](https://github.com/soltysh)) 2428 * Dual stack services (Phase II of IPv6DualStack feature) are enabled via the IPVS proxier. iptables proxier does not support dualstack yet. Dualstack iptables proxier is WIP and should catchup soon. ([#82091](https://github.com/kubernetes/kubernetes/pull/82091), [@khenidak](https://github.com/khenidak)) 2429 * to enable, kube-proxy must be have the following flags: 2430 * --proxy-mode=ipvs 2431 * --cluster-cidrs=<cidr>,<cidr> 2432 * The apiserver now uses http/1.1 to communicate with admission webhooks, opening multiple connections to satisfy concurrent requests, and allowing spreading requests across multiple backing pods. ([#82090](https://github.com/kubernetes/kubernetes/pull/82090), [@liggitt](https://github.com/liggitt)) 2433 * Added support to specify a global-access annotation for gce ILB. ([#81549](https://github.com/kubernetes/kubernetes/pull/81549), [@prameshj](https://github.com/prameshj)) 2434 * Added new startupProbe, related to KEP https://github.com/kubernetes/enhancements/issues/950. ([#77807](https://github.com/kubernetes/kubernetes/pull/77807), [@matthyx](https://github.com/matthyx)) 2435 * Adds \livez for liveness health checking for kube-apiserver. Using the parameter `--maximum-startup-sequence-duration` will allow the liveness endpoint to defer boot-sequence failures for the specified duration period. ([#81969](https://github.com/kubernetes/kubernetes/pull/81969), [@logicalhan](https://github.com/logicalhan)) 2436 * Server-side apply is now Beta. ([#81956](https://github.com/kubernetes/kubernetes/pull/81956), [@apelisse](https://github.com/apelisse)) 2437 * The `rejected` label in `apiserver_admission_webhook_admission_duration_seconds` metrics now properly indicates if a request was rejected. Add a new counter metrics `apiserver_admission_webhook_rejection_count` with details about the causing for a webhook rejection. ([#81399](https://github.com/kubernetes/kubernetes/pull/81399), [@roycaihw](https://github.com/roycaihw)) 2438 * Add `container_state` label to `running_container_count` kubelet metrics, to get count of containers based on their state(running/exited/created/unknown) ([#81573](https://github.com/kubernetes/kubernetes/pull/81573), [@irajdeep](https://github.com/irajdeep)) 2439 * Fix a bug in CRD openapi controller that user-defined CRD can overwrite OpenAPI definition/path for the CRD API. ([#81436](https://github.com/kubernetes/kubernetes/pull/81436), [@roycaihw](https://github.com/roycaihw)) 2440 * Service account tokens now include the JWT Key ID field in their header. ([#78502](https://github.com/kubernetes/kubernetes/pull/78502), [@ahmedtd](https://github.com/ahmedtd)) 2441 * Adds EndpointSlice integration to kube-proxy, can be enabled with EndpointSlice feature gate. ([#81430](https://github.com/kubernetes/kubernetes/pull/81430), [@robscott](https://github.com/robscott)) 2442 * Azure supports IPv6 only on ELB not ILB. The cloud provider will return an error if the service is internal and is IPv6. ([#80485](https://github.com/kubernetes/kubernetes/pull/80485), [@khenidak](https://github.com/khenidak)) 2443 * Notes on LB name: 2444 * to ensure backword and forward compat: 2445 * - SingleStack -v4 (pre v1.16) => BackendPool name == clusterName 2446 * - SingleStack -v6 => BackendPool name == clusterName (all cluster bootstrap uses this name) 2447 * DualStack: 2448 * => IPv4 BackendPool name == clusterName 2449 * => IPv6 BackendPool name == <clusterName>-IPv6 2450 * This result into: 2451 * - clusters moving from IPv4 to duakstack will require no changes 2452 * - clusters moving from IPv6 (while not seen in the wild, we can not rule out their existence) to dualstack will require deleting backend pools (the reconciler will take care of creating correct backendpools) 2453 * Promotes VolumePVCDataSource (Cloning) feature to beta for 1.16 release ([#81792](https://github.com/kubernetes/kubernetes/pull/81792), [@j-griffith](https://github.com/j-griffith)) 2454 * Remove kubectl log, use kubectl logs instead ([#78098](https://github.com/kubernetes/kubernetes/pull/78098), [@soltysh](https://github.com/soltysh)) 2455 * CSI ephemeral inline volume support is beta, i.e. the CSIInlineVolume feature gate is enabled by default ([#82004](https://github.com/kubernetes/kubernetes/pull/82004), [@pohly](https://github.com/pohly)) 2456 * kubectl: the --all-namespaces flag is now honored by `kubectl wait` ([#81468](https://github.com/kubernetes/kubernetes/pull/81468), [@ashutoshgngwr](https://github.com/ashutoshgngwr)) 2457 * Kube-proxy metrics are now marked as with the ALPHA stability level. ([#81626](https://github.com/kubernetes/kubernetes/pull/81626), [@logicalhan](https://github.com/logicalhan)) 2458 * Kube-controller-manager and cloud-controller-manager metrics are now marked as with the ALPHA stability level. ([#81624](https://github.com/kubernetes/kubernetes/pull/81624), [@logicalhan](https://github.com/logicalhan)) 2459 * Adds Endpoint Slice Controller for managing new EndpointSlice resource, disabled by default. ([#81048](https://github.com/kubernetes/kubernetes/pull/81048), [@robscott](https://github.com/robscott)) 2460 * + to run: ([#79386](https://github.com/kubernetes/kubernetes/pull/79386), [@khenidak](https://github.com/khenidak)) 2461 * Master: convert service CIDR to list `--service-cluster-ip-range=<CIDR>,<CIDR>` and make sure `IPv6DualStack` feature flag is turned on. The flag is validated and used as the following: 2462 * 1. `--service-cluster-ip-range[0]` is consider primary service range, and will be used for any service with `Service.Spec.IPFamily = nil` or any service in the at the time of turning on the feature flag. 2463 * 2. A cluster can be dualstack (i.e. Pods and nodes carry dualstack IPs) but does not need to support ingress on dualstack. In this case the cluster can perform egress using `PodIPs` (according to family and binding selection in user code) but will ingress will only be performed against the pod primary IP. This can be configured by supplying single entry to `--service-cluster-ip-range` flag. 2464 * 3. Maximum of two entries is allowed in `--service-cluster-ip-range` and they are validated to be dual stacked `i.e. --service-cluster-ip-range=<v4>,<v6> or --service-cluster-ip-range=<v6>,<v4>` 2465 * 4. Max 20 bit for range (min network bits `<v6>/108` or <v4>/12) 2466 * kube-controller-manager: convert service CIDR to list `--service-cluster-ip-range=<CIDR>,<CIDR>` and make sure `IPv6DualStack` feature flag is turned on. The flag is validated as above. 2467 * + to use: 2468 * A new service spec field `Service.Spec.IPFamily` has been added. The default of this field is family of (first service cidr in --service-cluster-ip-range flag). The value is defaulted as described above once the feature gate is turned on. Here are the possible values for this field: 2469 * 2. IPv4: api-server will assign an IP from a `service-cluster-ip-range` that is `ipv4` (either the primary or the secondary, according to how they were configured). 2470 * 2. IPv6: api-server will assign an IP from a `service-cluster-ip-range` that is `ipv6` (either the primary or the secondary, according to how they were configured). 2471 * Notes (v1.16): 2472 * 1. IPVS is the only proxy supported (as of v1.16 ) by Dualstack. 2473 * 2. Dualstack is mutually exclusive with `EndpointSlice` feature. They can not be turned on together. `metaproxy` is yet to implement EndpointSlice handling. 2474 * Kubelet metrics for /metrics and /metrics/probes are now marked as with the ALPHA stability level. ([#81534](https://github.com/kubernetes/kubernetes/pull/81534), [@logicalhan](https://github.com/logicalhan)) 2475 * Added metrics 'authentication_attempts' that can be used to understand the attempts of authentication. ([#81509](https://github.com/kubernetes/kubernetes/pull/81509), [@RainbowMango](https://github.com/RainbowMango)) 2476 * Fix in kube-proxy for SCTP nodeport service which only works for node's InternalIP, but doesn't work for other IPs present in the node when ipvs is enabled. ([#81477](https://github.com/kubernetes/kubernetes/pull/81477), [@paulsubrata55](https://github.com/paulsubrata55)) 2477 * The `CustomResourceValidation`, `CustomResourceSubresources`, `CustomResourceWebhookConversion` and `CustomResourcePublishOpenAPI` features are now GA, and the associated feature gates deprecated and will be removed in v1.18. ([#81965](https://github.com/kubernetes/kubernetes/pull/81965), [@roycaihw](https://github.com/roycaihw)) 2478 * Node-Problem-Detector v0.7.1 is used on GCI. ([#80726](https://github.com/kubernetes/kubernetes/pull/80726), [@wangzhen127](https://github.com/wangzhen127)) 2479 * kubeadm: prevent overriding of certain kubelet security configuration parameters if the user wished to modify them ([#81903](https://github.com/kubernetes/kubernetes/pull/81903), [@jfbai](https://github.com/jfbai)) 2480 * Introduce `node.kubernetes.io/exclude-balancer` and `node.kubernetes.io/exclude-disruption` labels in alpha to prevent cluster deployers from being dependent on the optional `node-role` labels which not all clusters may provide. ([#80238](https://github.com/kubernetes/kubernetes/pull/80238), [@smarterclayton](https://github.com/smarterclayton)) 2481 * Scheduler metrics are now marked as with the ALPHA stability level. ([#81576](https://github.com/kubernetes/kubernetes/pull/81576), [@logicalhan](https://github.com/logicalhan)) 2482 * cache-control headers are now set appropriately. Only openapi is cacheable if etags match. ([#81946](https://github.com/kubernetes/kubernetes/pull/81946), [@deads2k](https://github.com/deads2k)) 2483 * Added E2E tests validating WindowsOptions.RunAsUserName. ([#79539](https://github.com/kubernetes/kubernetes/pull/79539), [@bclau](https://github.com/bclau)) 2484 * Kube-apiserver metrics are now marked as with the ALPHA stability level. ([#81531](https://github.com/kubernetes/kubernetes/pull/81531), [@logicalhan](https://github.com/logicalhan)) 2485 * Move CSI volume expansion to beta. ([#81467](https://github.com/kubernetes/kubernetes/pull/81467), [@bertinatto](https://github.com/bertinatto)) 2486 * Support Kubelet plugin watcher on Windows nodes. ([#81397](https://github.com/kubernetes/kubernetes/pull/81397), [@ddebroy](https://github.com/ddebroy)) 2487 * Updates the requestedToCapacityRatioArguments to add resources parameter that allows the users to specify the resource name along with weights for each resource to score nodes based on the request to capacity ratio. ([#77688](https://github.com/kubernetes/kubernetes/pull/77688), [@sudeshsh](https://github.com/sudeshsh)) 2488 * Finalizer Protection for Service LoadBalancers is now in Beta (enabled by default). This feature ensures the Service resource is not fully deleted until the correlating load balancer resources are deleted. ([#81691](https://github.com/kubernetes/kubernetes/pull/81691), [@MrHohn](https://github.com/MrHohn)) 2489 * Adds support for vSphere volumes on Windows ([#80911](https://github.com/kubernetes/kubernetes/pull/80911), [@gab-satchi](https://github.com/gab-satchi)) 2490 * Log when kube-apiserver regenerates the OpenAPI spec and why. OpenAPI spec generation is a very CPU-heavy process that is sensitive to continuous updates of CRDs and APIServices. ([#81786](https://github.com/kubernetes/kubernetes/pull/81786), [@sttts](https://github.com/sttts)) 2491 * Added metrics aggregator_openapi_v2_regeneration_count, aggregator_openapi_v2_regeneration_gauge and apiextension_openapi_v2_regeneration_count metrics counting the triggering APIService and CRDs and the reason (add, update, delete). 2492 * Fix an issue with toleration merging & whitelist checking in the PodTolerationRestriction admission controller. ([#81732](https://github.com/kubernetes/kubernetes/pull/81732), [@tallclair](https://github.com/tallclair)) 2493 * Add a helper function to decode scheduler plugin args. ([#80696](https://github.com/kubernetes/kubernetes/pull/80696), [@hex108](https://github.com/hex108)) 2494 * Add metadata.generation=1 to old CustomResources. ([#82005](https://github.com/kubernetes/kubernetes/pull/82005), [@sttts](https://github.com/sttts)) 2495 * kubeadm no longer performs IPVS checks as part of its preflight checks ([#81791](https://github.com/kubernetes/kubernetes/pull/81791), [@yastij](https://github.com/yastij)) 2496 * The RemainingItemCount feature is now beta. ([#81682](https://github.com/kubernetes/kubernetes/pull/81682), [@caesarxuchao](https://github.com/caesarxuchao)) 2497 * remainingItemCount is the number of subsequent items in the list which are not included in this list response. If the list request contained label or field selectors, then the number of remaining items is unknown and the field will be left unset and omitted during serialization. If the list is complete (either because it is not chunking or because this is the last chunk), then there are no more remaining items and this field will be left unset and omitted during serialization. Servers older than v1.15 do not set this field. The intended use of the remainingItemCount is *estimating* the size of a collection. Clients should not rely on the remainingItemCount to be set or to be exact. 2498 * The CustomResourceDefaulting feature is promoted to beta and enabled by default. Defaults may be specified in structural schemas via the `apiextensions.k8s.io/v1` API. See https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#specifying-a-structural-schema for details. ([#81872](https://github.com/kubernetes/kubernetes/pull/81872), [@sttts](https://github.com/sttts)) 2499 * kubectl could scale custom resource again ([#81342](https://github.com/kubernetes/kubernetes/pull/81342), [@knight42](https://github.com/knight42)) 2500 * a CSI driver that supports ephemeral inline volumes must explicitly declare that by providing a CSIDriver object where the new "mode" field is set to "ephemeral" or "persistent+ephemeral" ([#80568](https://github.com/kubernetes/kubernetes/pull/80568), [@pohly](https://github.com/pohly)) 2501 * `framework.ExpectNoError` no longer logs the error and instead relies on using the new `log.Fail` as Gomega fail handler. ([#80253](https://github.com/kubernetes/kubernetes/pull/80253), [@pohly](https://github.com/pohly)) 2502 * Audit events now log the existence and patch of mutating webhooks. ([#77824](https://github.com/kubernetes/kubernetes/pull/77824), [@roycaihw](https://github.com/roycaihw)) 2503 * At Metadata audit level or higher, an annotation with key "mutation.webhook.admission.k8s.io/round_{round idx}_index_{order idx}" gets logged with JSON payload indicating a webhook gets invoked for given request and whether it mutated the object or not. 2504 * At Request audit level or higher, an annotation with key "patch.webhook.admission.k8s.io/round_{round idx}_index_{order idx}" get logged with the JSON payload logging the patch sent by a webhook for given request. 2505 * Resolves an issue that prevented block volumes from being resized. ([#81429](https://github.com/kubernetes/kubernetes/pull/81429), [@huffmanca](https://github.com/huffmanca)) 2506 * Verify that CRD default values in OpenAPI specs are pruned, with the exceptions of values under `metadata`. ([#78829](https://github.com/kubernetes/kubernetes/pull/78829), [@sttts](https://github.com/sttts)) 2507 * Use PostFilter instead of Postfilter in the scheduling framework ([#81800](https://github.com/kubernetes/kubernetes/pull/81800), [@draveness](https://github.com/draveness)) 2508 * Use PreFilter instead of Prefilter in the scheduling framework 2509 * Use PreBind instead of Prebind in the scheduling framework 2510 * Fix `kubectl logs -f` for windows server containers. ([#81747](https://github.com/kubernetes/kubernetes/pull/81747), [@Random-Liu](https://github.com/Random-Liu)) 2511 * fix azure disk naming matching issue due to case sensitive comparison ([#81720](https://github.com/kubernetes/kubernetes/pull/81720), [@andyzhangx](https://github.com/andyzhangx)) 2512 * Fixes a bug that when there is a "connection refused" error, the reflector's ListAndWatch func will return directly but what expected is that sleep 1 second and rewatch since the specified resourceVersion. ([#81634](https://github.com/kubernetes/kubernetes/pull/81634), [@likakuli](https://github.com/likakuli)) 2513 * Fixed a bug with the openAPI definition for io.k8s.apimachinery.pkg.runtime.RawExtension, which previously required a field "raw" to be specified ([#80773](https://github.com/kubernetes/kubernetes/pull/80773), [@jennybuckley](https://github.com/jennybuckley)) 2514 * kubeadm: print the stack trace of an error for klog level --v>=5 ([#80937](https://github.com/kubernetes/kubernetes/pull/80937), [@neolit123](https://github.com/neolit123)) 2515 * Fixes a problem with the iptables proxy mode that could result in long delays ([#80368](https://github.com/kubernetes/kubernetes/pull/80368), [@danwinship](https://github.com/danwinship)) 2516 * updating Service/Endpoints IPs in very large clusters on RHEL/CentOS 7. 2517 * kubeadm: support any Linux kernel version newer than 3.10 ([#81623](https://github.com/kubernetes/kubernetes/pull/81623), [@neolit123](https://github.com/neolit123)) 2518 * Added a metric 'apiserver_watch_events_sizes' that can be used to estimate sizes of watch events in the system. ([#80477](https://github.com/kubernetes/kubernetes/pull/80477), [@mborsz](https://github.com/mborsz)) 2519 * NormalizeScore plugin set is removed from scheduler framework config API. Use ScorePlugin only. ([#80930](https://github.com/kubernetes/kubernetes/pull/80930), [@liu-cong](https://github.com/liu-cong)) 2520 * kubeadm reset: unmount directories under "/var/lib/kubelet" for linux only ([#81494](https://github.com/kubernetes/kubernetes/pull/81494), [@Klaven](https://github.com/Klaven)) 2521 * updates fluentd-elasticsearch docker image to fluentd 1.6.3 ([#80912](https://github.com/kubernetes/kubernetes/pull/80912), [@monotek](https://github.com/monotek)) 2522 * Kubeadm now seamlessly migrates the CoreDNS Configuration when upgrading CoreDNS. ([#78033](https://github.com/kubernetes/kubernetes/pull/78033), [@rajansandeep](https://github.com/rajansandeep)) 2523 * Introduce support for applying pod overhead to pod cgroups, if the PodOverhead feature is enabled. ([#79247](https://github.com/kubernetes/kubernetes/pull/79247), [@egernst](https://github.com/egernst)) 2524 * Windows nodes on GCE now run with Windows Defender enabled. ([#81625](https://github.com/kubernetes/kubernetes/pull/81625), [@pjh](https://github.com/pjh)) 2525 2526 2527 2528 # v1.16.0-beta.1 2529 2530 [Documentation](https://docs.k8s.io) 2531 2532 ## Downloads for v1.16.0-beta.1 2533 2534 2535 filename | sha512 hash 2536 -------- | ----------- 2537 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes.tar.gz) | `16513ebb52b01afee26156dcd4c449455dc328d7a080ba54b3f3a4584dbd9297025e33a9dafe758b259ae6e33ccb84a18038f6f415e98be298761c4d3dfee94b` 2538 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-src.tar.gz) | `3933f441ebca812835d6f893ec378896a8adb7ae88ca53247fa402aee1fda00d533301ac806f6bf106badf2f91be8c2524fd98e9757244b4b597c39124c59d01` 2539 2540 ### Client Binaries 2541 2542 filename | sha512 hash 2543 -------- | ----------- 2544 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-darwin-386.tar.gz) | `28f0a8979f956aa5b3be1c1158a3ade1b242aac332696cb604fbdba44c4279caa1008840af01e50692bf48d0342018f882dd6e30f9fe3279e9784094cfc9ff3c` 2545 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-darwin-amd64.tar.gz) | `8804f60b690e5180125cf6ac6d739ad5432b364c5e0d0ee0d2f06220c86ca3a2cffc475e0e3c46c19466e5d1566a5b8bf0a33191cba5bbd3ff27ac64ceee57a0` 2546 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-linux-386.tar.gz) | `8f7f86db5a496afd269b926b6baf341bbd4208f49b48fad1a44c5424812667b3bd7912b5b97bd7844dee2a7c6f9441628f7b5db3caa14429020de7788289191c` 2547 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-linux-amd64.tar.gz) | `7407dc1216cac39f15ca9f75be47c0463a151a3fda7d9843a67c0043c69858fb36eaa6b4194ce5cefd125acd7f521c4b958d446bb0c95ca73a3b3ae47af2c3ee` 2548 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-linux-arm.tar.gz) | `249a82a0af7d8062f49edd9221b3823590b6d166c1bca12c787ae640d6a131bd6a3d7c99136de62074afa6cabe8900dcf4e11037ddbfdf9d5252fc16e256eeb5` 2549 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-linux-arm64.tar.gz) | `3a8416d99b6ae9bb6d568ff15d1783dc521fe58c60230f38126c64a7739bf03d8490a9a10042d1c4ef07290eaced6cb9d42a9728d4b937305d63f8d3cc7a66f8` 2550 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-linux-ppc64le.tar.gz) | `105bf4afeccf0b314673265b969d1a7f3796ca3098afa788c43cd9ff3e14ee409392caa5766631cca180e790d92731a48f5e7156167637b97abc7c178dd390f3` 2551 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-linux-s390x.tar.gz) | `98de73accb7deba9896e14a5012a112f6fd00d6e6868e4d21f61b06605efa8868f1965a1c1ba72bb8847416bc789bd7ef5c1a125811b6c6df060217cd84fdb2c` 2552 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-windows-386.tar.gz) | `7a43f3285b0ab617990497d41ceadfbd2be2b72d433b02508c198e9d380fb5e0a96863cc14d0e9bf0317df13810af1ab6b7c47cd4fa1d0619a00c9536dc60f0f` 2553 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-client-windows-amd64.tar.gz) | `f3fafcffc949bd7f8657dd684c901e199b21c4812009aca1f8cf3c8bf3c3230cab072208d3702d7a248c0b957bc513306dd437fb6a54e1e64b4d7dc8c3c180cd` 2554 2555 ### Server Binaries 2556 2557 filename | sha512 hash 2558 -------- | ----------- 2559 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-server-linux-amd64.tar.gz) | `87b46e73ae2162ee49f510da6549e57503d3ea94b3c4488f39b0b93d45603f540ece30c3784c5e201711a7ddd1260481cd20ac4c618eaf46879e841d054a115a` 2560 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-server-linux-arm.tar.gz) | `80ba8e615497c0b9c339fbd2d6a4dda54fdbd5659abd7d8e8d448d8d8c24ba7f0ec48693e4bf8ed20513c46432f2a0f1039ab9044f0ed006b935a58772372d95` 2561 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-server-linux-arm64.tar.gz) | `b4a76a5fc026b4b3b5f9666df05e46896220591b21c147982ff3d91cec7330ed78cf1fc63f5ab759820aadbcfe400c1ad75d5151d9217d42e3da5873e0ff540d` 2562 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-server-linux-ppc64le.tar.gz) | `fb435dfd5514e4cd3bc16b9e71865bff3cdd5123fc272c8cbc5956c260449e0dcfd30d2fdb120da73134e62f48507c5a02d4528d7b9d978765ff4ed740b274e8` 2563 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-server-linux-s390x.tar.gz) | `65ed3d372a4d03493d0a586c7f67f1236aa99f02552195f1fb58079bc24787200d9a0f34d0c311a846345d0d70d02ad726f74376a91d3ced234bbfdce80c5133` 2564 2565 ### Node Binaries 2566 2567 filename | sha512 hash 2568 -------- | ----------- 2569 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-node-linux-amd64.tar.gz) | `c9161689532a5e995a68bb0985a983dc43d8e747a05f37849cd33062c07e5202417b26bff652b8bc9c0005026618b7ebc56f918c71747a3addb5da044e683b4a` 2570 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-node-linux-arm.tar.gz) | `7dba9fdb290f33678983c046eb145446edb1b7479c2403f9e8bd835c3d832ab1f2acb28124c53af5b046d47ab433312d6a654f000a22f8e10795b0bc45bfbddb` 2571 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-node-linux-arm64.tar.gz) | `8c435824667cd9ec7efdfb72c1d060f62ca61b285cbb9575a6e6013e20ec5b379f77f51d43ae21c1778a3eb3ef69df8895213c54e4b9f39c67c929a276be12de` 2572 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-node-linux-ppc64le.tar.gz) | `2cfca30dbe49a38cd1f3c78135f60bf7cb3dae0a8ec5d7fa651e1c5949254876fbab8a724ed9a13f733a85b9960edcc4cc971dc3c16297db609209c4270f144f` 2573 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-node-linux-s390x.tar.gz) | `63bbe469ddd1be48624ef5627fef1e1557a691819c71a77d419d59d101e8e6ee391eb8545da35b412b94974c06d73329a13660484ab26087a178f34a827a3dcb` 2574 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-beta.1/kubernetes-node-windows-amd64.tar.gz) | `07cb97d5a3b7d0180a9e22696f417422a0c043754c81ae68338aab7b520aa7c119ff53b9ad835f9a0bc9ea8c07483ce506af48d65641dd15d30209a696b064bb` 2575 2576 ## Changelog since v1.16.0-alpha.3 2577 2578 ### Action Required 2579 2580 * scheduler.alpha.kubernetes.io/critical-pod annotation is removed. Pod priority (spec.priorityClassName) should be used instead to mark pods as critical. Action required! ([#80342](https://github.com/kubernetes/kubernetes/pull/80342), [@draveness](https://github.com/draveness)) 2581 * Removed cadvisor metric labels `pod_name` and `container_name` to match instrumentation guidelines. ([#80376](https://github.com/kubernetes/kubernetes/pull/80376), [@ehashman](https://github.com/ehashman)) 2582 * Action required: any Prometheus queries that match `pod_name` and `container_name` labels (e.g. cadvisor or kubelet probe metrics) must be updated to use `pod` and `container` instead. 2583 * Remove DirectCodecFactory(replace with serializer.WithoutConversionCodecFactory), DirectEncoder(replace with runtime.WithVersionEncoder) and DirectDecoder(replace with runtime.WithoutVersionDecoder). action required ([#79263](https://github.com/kubernetes/kubernetes/pull/79263), [@draveness](https://github.com/draveness)) 2584 2585 ### Other notable changes 2586 2587 * fix: detach azure disk issue using dangling error ([#81266](https://github.com/kubernetes/kubernetes/pull/81266), [@andyzhangx](https://github.com/andyzhangx)) 2588 * Conversion webhooks can now indicate they support receiving and responding with `ConversionReview` API objects in the `apiextensions.k8s.io/v1` version by including `v1` in the `conversionReviewVersions` list in their CustomResourceDefinition. Conversion webhooks must respond with a ConversionReview object in the same apiVersion they receive. `apiextensions.k8s.io/v1` `ConversionReview` responses must specify a `response.uid` that matches the `request.uid` of the object they were sent. ([#81476](https://github.com/kubernetes/kubernetes/pull/81476), [@liggitt](https://github.com/liggitt)) 2589 * The `CustomResourceDefinition` API type is promoted to `apiextensions.k8s.io/v1` with the following changes: ([#79604](https://github.com/kubernetes/kubernetes/pull/79604), [@liggitt](https://github.com/liggitt)) 2590 * Use of the new `default` feature in validation schemas is limited to v1 2591 * `spec.scope` is no longer defaulted to `Namespaced` and must be explicitly specified 2592 * `spec.version` is removed; use `spec.versions` instead 2593 * `spec.validation` is removed; use `spec.versions[*].schema` instead 2594 * `spec.subresources` is removed; use `spec.versions[*].subresources` instead 2595 * `spec.additionalPrinterColumns` is removed; use `spec.versions[*].additionalPrinterColumns` instead 2596 * `spec.conversion.webhookClientConfig` is moved to `spec.conversion.webhook.clientConfig` 2597 * `spec.conversion.conversionReviewVersions` is moved to `spec.conversion.webhook.conversionReviewVersions` 2598 * `spec.versions[*].schema.openAPIV3Schema` is now required when creating v1 CustomResourceDefinitions 2599 * `spec.preserveUnknownFields: true` is disallowed when creating v1 CustomResourceDefinitions; it must be specified within schema definitions as `x-kubernetes-preserve-unknown-fields: true` 2600 * In `additionalPrinterColumns` items, the `JSONPath` field was renamed to `jsonPath` (fixes https://github.com/kubernetes/kubernetes/issues/66531) 2601 * openapi now advertises correctly supported patch types for custom resources ([#81515](https://github.com/kubernetes/kubernetes/pull/81515), [@liggitt](https://github.com/liggitt)) 2602 * Kubelet could be run with no Azure identity without subscriptionId configured now. ([#81500](https://github.com/kubernetes/kubernetes/pull/81500), [@feiskyer](https://github.com/feiskyer)) 2603 * A sample cloud provider configure is: '{"vmType": "vmss", "useInstanceMetadata": true}'. 2604 * Volumes specified in a pod but not used in it are no longer unnecessarily formatted, mounted and reported in `node.status.volumesInUse`. ([#81163](https://github.com/kubernetes/kubernetes/pull/81163), [@jsafrane](https://github.com/jsafrane)) 2605 * kubeadm: use etcd's /health endpoint for a HTTP liveness probe on localhost instead of having a custom health check using etcdctl ([#81385](https://github.com/kubernetes/kubernetes/pull/81385), [@neolit123](https://github.com/neolit123)) 2606 * kubeamd: use the --pod-network-cidr flag to init or use the podSubnet field in the kubeadm config to pass a comma separated list of pod CIDRs. ([#79033](https://github.com/kubernetes/kubernetes/pull/79033), [@Arvinderpal](https://github.com/Arvinderpal)) 2607 * Update to use go 1.12.9 ([#81489](https://github.com/kubernetes/kubernetes/pull/81489), [@BenTheElder](https://github.com/BenTheElder)) 2608 * Update Azure SDK + go-autorest API versions ([#79574](https://github.com/kubernetes/kubernetes/pull/79574), [@justaugustus](https://github.com/justaugustus)) 2609 * Extender bind should respect IsInterested ([#79804](https://github.com/kubernetes/kubernetes/pull/79804), [@yqwang-ms](https://github.com/yqwang-ms)) 2610 * Add instruction to setup "Application Default Credentials" to run GCE Windows e2e tests locally. ([#81337](https://github.com/kubernetes/kubernetes/pull/81337), [@YangLu1031](https://github.com/YangLu1031)) 2611 * Scheduler should terminate when it looses leader lock. ([#81306](https://github.com/kubernetes/kubernetes/pull/81306), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) 2612 * kubelet now exports an "kubelet_evictions" metric that counts the number of pod evictions carried out by the kubelet to reclaim resources ([#81377](https://github.com/kubernetes/kubernetes/pull/81377), [@sjenning](https://github.com/sjenning)) 2613 * Return error when the scoring plugin returns score out of range [0, 100]. ([#81015](https://github.com/kubernetes/kubernetes/pull/81015), [@draveness](https://github.com/draveness)) 2614 * Update to use go 1.12.8 ([#81390](https://github.com/kubernetes/kubernetes/pull/81390), [@cblecker](https://github.com/cblecker)) 2615 * kube-proxy --cleanup will return the correct exit code if the cleanup was successful ([#78775](https://github.com/kubernetes/kubernetes/pull/78775), [@johscheuer](https://github.com/johscheuer)) 2616 * remove iSCSI volume storage cleartext secrets in logs ([#81215](https://github.com/kubernetes/kubernetes/pull/81215), [@zouyee](https://github.com/zouyee)) 2617 * Use a named array instead of a score array in normalizing-score phase. ([#80901](https://github.com/kubernetes/kubernetes/pull/80901), [@draveness](https://github.com/draveness)) 2618 * If scheduler extender filtered a not found node, current scheduling round for this pod will just be skipped. ([#79641](https://github.com/kubernetes/kubernetes/pull/79641), [@yqwang-ms](https://github.com/yqwang-ms)) 2619 * Update golang/x/net dependency to bring in fixes for CVE-2019-9512, CVE-2019-9514 ([#81394](https://github.com/kubernetes/kubernetes/pull/81394), [@cblecker](https://github.com/cblecker)) 2620 * Fixes CVE-2019-11250: client-go header logging (at verbosity levels >= 7) now masks `Authorization` header contents ([#81330](https://github.com/kubernetes/kubernetes/pull/81330), [@tedyu](https://github.com/tedyu)) 2621 * Resolves a transient 404 response to custom resource requests during server startup ([#81244](https://github.com/kubernetes/kubernetes/pull/81244), [@liggitt](https://github.com/liggitt)) 2622 * Non nil DataSource entries on PVC's are now displayed as part of `describe pvc` output. ([#76463](https://github.com/kubernetes/kubernetes/pull/76463), [@j-griffith](https://github.com/j-griffith)) 2623 * Fix Azure client requests stuck issues on http.StatusTooManyRequests (HTTP Code 429). ([#81279](https://github.com/kubernetes/kubernetes/pull/81279), [@feiskyer](https://github.com/feiskyer)) 2624 * Implement a new feature that allows applying kustomize patches to static pod manifests generated by kubeadm. ([#80905](https://github.com/kubernetes/kubernetes/pull/80905), [@fabriziopandini](https://github.com/fabriziopandini)) 2625 * Add a service annotation `service.beta.kubernetes.io/azure-pip-name` to specify the public IP name for Azure load balancer. ([#81213](https://github.com/kubernetes/kubernetes/pull/81213), [@nilo19](https://github.com/nilo19)) 2626 * Fix a bug in the IPVS proxier where virtual servers are not cleaned up even though the corresponding Service object was deleted. ([#80942](https://github.com/kubernetes/kubernetes/pull/80942), [@gongguan](https://github.com/gongguan)) 2627 * Add scheduling support for RuntimeClasses. RuntimeClasses can now specify nodeSelector constraints & tolerations, which are merged into the PodSpec for pods using that RuntimeClass. ([#80825](https://github.com/kubernetes/kubernetes/pull/80825), [@tallclair](https://github.com/tallclair)) 2628 * etcd Docker image can be run as non-root ([#79722](https://github.com/kubernetes/kubernetes/pull/79722), [@randomvariable](https://github.com/randomvariable)) 2629 * kubeadm: the permissions of generated CSR files are changed from 0644 to 0600 ([#81217](https://github.com/kubernetes/kubernetes/pull/81217), [@SataQiu](https://github.com/SataQiu)) 2630 * Fix conflicted cache when the requests are canceled by other Azure operations. ([#81282](https://github.com/kubernetes/kubernetes/pull/81282), [@feiskyer](https://github.com/feiskyer)) 2631 * Fix kubelet NodeLease potential performance issues. Kubelet now will try to update lease using cached one instead of get from API Server every time. ([#81174](https://github.com/kubernetes/kubernetes/pull/81174), [@answer1991](https://github.com/answer1991)) 2632 * Improves validation errors for custom resources ([#81212](https://github.com/kubernetes/kubernetes/pull/81212), [@liggitt](https://github.com/liggitt)) 2633 * Improvement in Kube-proxy. Kube-proxy waits for some duration for the node to be defined. ([#77167](https://github.com/kubernetes/kubernetes/pull/77167), [@paulsubrata55](https://github.com/paulsubrata55)) 2634 * hyperkube will drop support for cloud-controller-manager in a future release ([#81219](https://github.com/kubernetes/kubernetes/pull/81219), [@dims](https://github.com/dims)) 2635 * added an new Prometheus counter metric "sync_proxy_rules_iptables_restore_failures_total" for kube-proxy iptables-restore failures (both ipvs and iptables modes) ([#81210](https://github.com/kubernetes/kubernetes/pull/81210), [@figo](https://github.com/figo)) 2636 * Add a `Patch` method to `ScaleInterface` ([#80699](https://github.com/kubernetes/kubernetes/pull/80699), [@knight42](https://github.com/knight42)) 2637 * switch to VM Update call in attach/detach disk operation, original CreateOrUpdate call may lead to orphaned VMs or blocked resources ([#81208](https://github.com/kubernetes/kubernetes/pull/81208), [@andyzhangx](https://github.com/andyzhangx)) 2638 * Add a azure cloud configuration `LoadBalancerName` and `LoadBalancerResourceGroup` to allow the corresponding customizations of azure load balancer. ([#81054](https://github.com/kubernetes/kubernetes/pull/81054), [@nilo19](https://github.com/nilo19)) 2639 * Update the GCE windows node image to include hot fixes since July. ([#81106](https://github.com/kubernetes/kubernetes/pull/81106), [@YangLu1031](https://github.com/YangLu1031)) 2640 * Kubelet considers all static pods as critical. Static pods pass kubelet admission even if a node does not have enough resources. Users must ensure that they account for resources when creating static pods. ([#80491](https://github.com/kubernetes/kubernetes/pull/80491), [@hpandeycodeit](https://github.com/hpandeycodeit)) 2641 * kube-apiserver: the `--basic-auth-file` flag and authentication mode is deprecated and will be removed in a future release. It is not recommended for production environments. ([#81152](https://github.com/kubernetes/kubernetes/pull/81152), [@tedyu](https://github.com/tedyu)) 2642 * Fix a bug that pretty printer marshals empty byte or uint8 slice as null ([#81096](https://github.com/kubernetes/kubernetes/pull/81096), [@roycaihw](https://github.com/roycaihw)) 2643 * Deprecate the `--cloud-provider-gce-lb-src-cidrs` flag in the kube-apiserver. This flag will be removed once the GCE Cloud Provider is removed from kube-apiserver. ([#81094](https://github.com/kubernetes/kubernetes/pull/81094), [@andrewsykim](https://github.com/andrewsykim)) 2644 * cloud-controller-manager binaries and docker images are no longer shipped with kubernetes releases. ([#81029](https://github.com/kubernetes/kubernetes/pull/81029), [@dims](https://github.com/dims)) 2645 * API: the metadata.selfLink field is deprecated in individual and list objects. It will no longer be returned starting in v1.20, and the field will be removed entirely in v1.21. ([#80978](https://github.com/kubernetes/kubernetes/pull/80978), [@wojtek-t](https://github.com/wojtek-t)) 2646 2647 2648 2649 # v1.16.0-alpha.3 2650 2651 [Documentation](https://docs.k8s.io) 2652 2653 ## Downloads for v1.16.0-alpha.3 2654 2655 2656 filename | sha512 hash 2657 -------- | ----------- 2658 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes.tar.gz) | `82bc119f8d1e44518ab4f4bdefb96158b1a3634c003fe1bc8dcd62410189449fbd6736126409d39a6e2d211a036b4aa98baef3b3c6d9f7505e63430847d127c2` 2659 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-src.tar.gz) | `bbf330b887a5839e3d3219f5f4aa38f1c70eab64228077f846da80395193b2b402b60030741de14a9dd4de963662cfe694f6ab04035309e54dc48e6dddd5c05d` 2660 2661 ### Client Binaries 2662 2663 filename | sha512 hash 2664 -------- | ----------- 2665 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-darwin-386.tar.gz) | `8d509bdc1ca62463cbb25548ec270792630f6a883f3194e5bdbbb3d6f8568b00f695e39950b7b01713f2f05f206c4d1df1959c6ee80f8a3e390eb94759d344b2` 2666 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-darwin-amd64.tar.gz) | `1b00b3a478c210e3c3e6c346f5c4f7f43a00d5ef6acb8d9c1feaf26f913b9d4f97eb6db99bbf67953ef6399abe4fbb79324973c1744a6a8cd76067cb2aeed2ca` 2667 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-linux-386.tar.gz) | `82424207b4ef52c3722436eaaf86dbe5c93c6670fd09c2b04320251028fd1bb75724b4f490b6e8b443bd8e5f892ab64612cd22206119924dafde424bdee9348a` 2668 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-linux-amd64.tar.gz) | `57ba937e58755d3b7dfd19626fedb95718f9c1d44ac1c5b4c8c46d11ba0f8783f3611c7b946b563cac9a3cf104c35ba5605e5e76b48ba2a707d787a7f50f7027` 2669 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-linux-arm.tar.gz) | `3a3601026e019b299a6f662b887ebe749f08782d7ed0d37a807c38a01c6ba19f23e2837c9fb886053ad6e236a329f58a11ee3ec4ba96a8729905ae78a7f6c58c` 2670 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-linux-arm64.tar.gz) | `4cdeb2e678c6b817a04f9f5d92c5c6df88e0f954550961813fca63af4501d04c08e3f4353dd8b6dce96e2ee197a4c688245f03c888417a436b3cf70abd4ba53a` 2671 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-linux-ppc64le.tar.gz) | `0cc7c8f7b48f5affb679352a94e42d8b4003b9ca6f8cbeaf315d2eceddd2e8446a58ba1d4a0df18e8f9c69d0d3b5a46f25b2e6a916e57975381e504d1a4daa1b` 2672 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-linux-s390x.tar.gz) | `9d8fa639f543e707dc65f24ce2f8c73a50c606ec7bc27d17840f45ac150d00b3b3f83de5e3b21f72b598acf08273e4b9a889f199f4ce1b1d239b28659e6cd131` 2673 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-windows-386.tar.gz) | `05bf6e696da680bb8feec4f411f342a9661b6165f4f0c72c069871983f199418c4d4fa1e034136bc8be41c5fecc9934a123906f2d5666c09a876db16ae8c11ad` 2674 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-client-windows-amd64.tar.gz) | `b2097bc851f5d3504e562f68161910098b46c66c726b92b092a040acda965fed01f45e7b9e513a4259c7a5ebd65d7aa3e3b711f4179139a935720d91216ef5c2` 2675 2676 ### Server Binaries 2677 2678 filename | sha512 hash 2679 -------- | ----------- 2680 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-server-linux-amd64.tar.gz) | `721bd09b64e5c8f220332417089a772d9073c0dc5cdfa240984cfeb0d681b4a02620fb3ebf1b9f6a82a4dd3423f5831c259d4bad502dce87f145e0a08cb73ee9` 2681 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-server-linux-arm.tar.gz) | `e7638ce4b88b4282f0a157593cfe809fa9cc9139ea7ebae4762ef5ac1dfaa516903a8acb34a45937eb94b2699e5d4c68c639cbe40cbed2a6b97681aeace9948e` 2682 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-server-linux-arm64.tar.gz) | `395566c4be3c2ca5b07e81221b3370bc7ccbef0879f96a9384650fcaf4f699f3b2744ba1d97ae42cc6c5d9e1a65a41a793a8b0c9e01a0a65f57c56b1420f8141` 2683 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-server-linux-ppc64le.tar.gz) | `90fcba066efd76d2f271a0eb26ed4d90483674d04f5e8cc39ec1e5b7f343311f2f1c40de386f35d3c69759628a1c7c075559c09b6c4542e42fbbe0daeb61a5fa` 2684 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-server-linux-s390x.tar.gz) | `b25014bcf4138722a710451f6e58ee57588b4d47fcceeda8f6866073c1cc08641082ec56e94b0c6d586c0835ce9b55d205d254436fc22a744b24d8c74e8e5cce` 2685 2686 ### Node Binaries 2687 2688 filename | sha512 hash 2689 -------- | ----------- 2690 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-node-linux-amd64.tar.gz) | `6925a71096530f7114a68e755d07cb8ba714bc60b477360c85d76d7b71d3a3c0b78a650877d81aae35b308ded27c8207b5fe72d990abc43db3aa8a7d6d7f94f4` 2691 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-node-linux-arm.tar.gz) | `073310e1ccf9a8af998d4c0402ae86bee4f253d2af233b0c45cea55902268c2fe7190a41a990b079e24536e9efa27b94249c3a9236531a166ba3ac06c0f26f92` 2692 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-node-linux-arm64.tar.gz) | `c55e9aecef906e56a6003f441a7d336846edb269aed1c7a31cf834b0730508706e73ea0ae135c1604b0697c9e2582480fbfba8ba105152698c240e324da0cbd2` 2693 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-node-linux-ppc64le.tar.gz) | `e89d72d27bb0a7f9133ef7310f455ba2b4c46e9852c43e0a981b68a413bcdd18de7168eb16d93cf87a5ada6a4958592d3be80c9be1e6895fa48e2f7fa70f188d` 2694 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-node-linux-s390x.tar.gz) | `6ef8a25f2f80a806672057dc030654345e87d269babe7cf166f7443e04c0b3a9bc1928cbcf5abef1f0f0fcd37f3a727f789887dbbdae62f9d1fd90a71ed26b39` 2695 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.3/kubernetes-node-windows-amd64.tar.gz) | `22fd1cea6e0150c06dbdc7249635bbf93c4297565d5a9d13e653f9365cd61a0b8306312efc806d267c47be81621016b114510a269c622cccc916ecff4d10f33c` 2696 2697 ## Changelog since v1.16.0-alpha.2 2698 2699 ### Action Required 2700 2701 * ACTION REQUIRED: ([#80676](https://github.com/kubernetes/kubernetes/pull/80676), [@fabriziopandini](https://github.com/fabriziopandini)) 2702 * kubeadm now deletes the bootstrap-kubelet.conf file after TLS bootstrap 2703 * User relying on bootstrap-kubelet.conf should switch to kubelet.conf that contains node credentials 2704 2705 ### Other notable changes 2706 2707 * Fixes validation of VolumeAttachment API objects created with inline volume sources. ([#80945](https://github.com/kubernetes/kubernetes/pull/80945), [@tedyu](https://github.com/tedyu)) 2708 * Azure disks of shared kind will no longer fail if they do not contain skuname or ([#80837](https://github.com/kubernetes/kubernetes/pull/80837), [@rmweir](https://github.com/rmweir)) 2709 * storageaccounttype. 2710 * kubeadm: fix "certificate-authority" files not being pre-loaded when using file discovery ([#80966](https://github.com/kubernetes/kubernetes/pull/80966), [@neolit123](https://github.com/neolit123)) 2711 * Errors from pod volume set up are now propagated as pod events. ([#80369](https://github.com/kubernetes/kubernetes/pull/80369), [@jsafrane](https://github.com/jsafrane)) 2712 * kubeadm: enable secure serving for the kube-scheduler ([#80951](https://github.com/kubernetes/kubernetes/pull/80951), [@neolit123](https://github.com/neolit123)) 2713 * Kubernetes client users may disable automatic compression when invoking Kubernetes APIs by setting the `DisableCompression` field on their rest.Config. This is recommended when clients communicate primarily over high bandwidth / low latency networks where response compression does not improve end to end latency. ([#80919](https://github.com/kubernetes/kubernetes/pull/80919), [@smarterclayton](https://github.com/smarterclayton)) 2714 * kubectl get did not correctly count the number of binaryData keys when listing config maps. ([#80827](https://github.com/kubernetes/kubernetes/pull/80827), [@smarterclayton](https://github.com/smarterclayton)) 2715 * Implement "post-filter" extension point for scheduling framework ([#78097](https://github.com/kubernetes/kubernetes/pull/78097), [@draveness](https://github.com/draveness)) 2716 * Reduces GCE PD Node Attach Limits by 1 since the node boot disk is considered an attachable disk ([#80923](https://github.com/kubernetes/kubernetes/pull/80923), [@davidz627](https://github.com/davidz627)) 2717 * This PR fixes an error when using external etcd but storing etcd certificates in the same folder and with the same name used by kubeadm for local etcd certificates; for an older version of kubeadm, the workaround is to avoid file name used by kubeadm for local etcd. ([#80867](https://github.com/kubernetes/kubernetes/pull/80867), [@fabriziopandini](https://github.com/fabriziopandini)) 2718 * When specifying `--(kube|system)-reserved-cgroup`, with `--cgroup-driver=systemd`, it is now possible to use the fully qualified cgroupfs name (i.e. `/test-cgroup.slice`). ([#78793](https://github.com/kubernetes/kubernetes/pull/78793), [@mattjmcnaughton](https://github.com/mattjmcnaughton)) 2719 * kubeadm: treat non-fatal errors as warnings when doing reset ([#80862](https://github.com/kubernetes/kubernetes/pull/80862), [@drpaneas](https://github.com/drpaneas)) 2720 * kube-addon-manager has been updated to v9.0.2 to fix a bug in leader election (https://github.com/kubernetes/kubernetes/pull/80575) ([#80861](https://github.com/kubernetes/kubernetes/pull/80861), [@mborsz](https://github.com/mborsz)) 2721 * Determine system model to get credentials for windows nodes. ([#80764](https://github.com/kubernetes/kubernetes/pull/80764), [@liyanhui1228](https://github.com/liyanhui1228)) 2722 * TBD ([#80730](https://github.com/kubernetes/kubernetes/pull/80730), [@jennybuckley](https://github.com/jennybuckley)) 2723 * The `AdmissionReview` API sent to and received from admission webhooks has been promoted to `admission.k8s.io/v1`. Webhooks can specify a preference for receiving `v1` AdmissionReview objects with `admissionReviewVersions: ["v1","v1beta1"]`, and must respond with an API object in the same `apiVersion` they are sent. When webhooks use `admission.k8s.io/v1`, the following additional validation is performed on their responses: ([#80231](https://github.com/kubernetes/kubernetes/pull/80231), [@liggitt](https://github.com/liggitt)) 2724 * `response.patch` and `response.patchType` are not permitted from validating admission webhooks 2725 * `apiVersion: "admission.k8s.io/v1"` is required 2726 * `kind: "AdmissionReview"` is required 2727 * `response.uid: "<value of request.uid>"` is required 2728 * `response.patchType: "JSONPatch"` is required (if `response.patch` is set) 2729 * "kubeadm join" fails if file-based discovery is too long, with a default timeout of 5 minutes. ([#80804](https://github.com/kubernetes/kubernetes/pull/80804), [@olivierlemasle](https://github.com/olivierlemasle)) 2730 * enhance Azure cloud provider code to support both AAD and ADFS authentication. ([#80841](https://github.com/kubernetes/kubernetes/pull/80841), [@rjaini](https://github.com/rjaini)) 2731 * Attempt to set the kubelet's hostname & internal IP if `--cloud-provider=external` and no node addresses exists ([#75229](https://github.com/kubernetes/kubernetes/pull/75229), [@andrewsykim](https://github.com/andrewsykim)) 2732 * kubeadm: avoid double deletion of the upgrade prepull DaemonSet ([#80798](https://github.com/kubernetes/kubernetes/pull/80798), [@xlgao-zju](https://github.com/xlgao-zju)) 2733 * Fixes problems with connecting to services on localhost on some systems; in particular, DNS queries to systemd-resolved on Ubuntu. ([#80591](https://github.com/kubernetes/kubernetes/pull/80591), [@danwinship](https://github.com/danwinship)) 2734 * Implement normalize plugin extension point for the scheduler framework. ([#80383](https://github.com/kubernetes/kubernetes/pull/80383), [@liu-cong](https://github.com/liu-cong)) 2735 * Fixed the bash completion error with override flags. ([#80802](https://github.com/kubernetes/kubernetes/pull/80802), [@dtaniwaki](https://github.com/dtaniwaki)) 2736 * Fix CVE-2019-11247: API server allows access to custom resources via wrong scope ([#80750](https://github.com/kubernetes/kubernetes/pull/80750), [@sttts](https://github.com/sttts)) 2737 * Failed iscsi logout is now re-tried periodically. ([#78941](https://github.com/kubernetes/kubernetes/pull/78941), [@jsafrane](https://github.com/jsafrane)) 2738 * Fix public IP not found issues for VMSS nodes ([#80703](https://github.com/kubernetes/kubernetes/pull/80703), [@feiskyer](https://github.com/feiskyer)) 2739 * In order to enable dual-stack support within kubeadm and kubernetes components, as part of the init config file, the user should set feature-gate IPv6DualStack=true in the ClusterConfiguration. Additionally, for each worker node, the user should set the feature-gate for kubelet using either nodeRegistration.kubeletExtraArgs or KUBELET_EXTRA_ARGS. ([#80531](https://github.com/kubernetes/kubernetes/pull/80531), [@Arvinderpal](https://github.com/Arvinderpal)) 2740 * Fix error in `kubeadm join --discovery-file` when using discovery files with embedded credentials ([#80675](https://github.com/kubernetes/kubernetes/pull/80675), [@fabriziopandini](https://github.com/fabriziopandini)) 2741 2742 2743 2744 # v1.16.0-alpha.2 2745 2746 [Documentation](https://docs.k8s.io) 2747 2748 ## Downloads for v1.16.0-alpha.2 2749 2750 2751 filename | sha512 hash 2752 -------- | ----------- 2753 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes.tar.gz) | `7dfa3f8b9e98e528e2b49ed9cca5e95f265b9e102faac636ff0c29e045689145be236b98406a62eb0385154dc0c1233cac049806c99c9e46590cad5aa729183f` 2754 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-src.tar.gz) | `7cf14b92c96cab5fcda3115ec66b44562ca26ea6aa46bc7fa614fa66bda1bdf9ac1f3c94ef0dfa0e37c992c7187ecf4205b253f37f280857e88a318f8479c9a9` 2755 2756 ### Client Binaries 2757 2758 filename | sha512 hash 2759 -------- | ----------- 2760 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-darwin-386.tar.gz) | `4871756de2cd1add0b07ec1e577c500d18a59e2f761595b939e1d4e10fbe0a119479ecaaf53d75cb2138363deae23cc88cba24fe3018cec6a27a3182f37cae92` 2761 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-darwin-amd64.tar.gz) | `dbd9ca5fd90652ffc1606f50029d711eb52d34b707b7c04f29201f85aa8a5081923a53585513634f3adb6ace2bc59be9d4ad2abc49fdc3790ef805378c111e68` 2762 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-linux-386.tar.gz) | `6b049098b1dc65416c5dcc30346b82e5cf69a1cdd7e7b065429a76d302ef4b2a1c8e2dc621e9d5c1a6395a1fbd97f196d99404810880d118576e7b94e5621e4c` 2763 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-linux-amd64.tar.gz) | `7240a9d49e445e9fb0c9d360a9287933c6c6e7d81d6e11b0d645d3f9b6f3f1372cc343f03d10026518df5d6c95525e84c41b06a034c9ec2c9e306323dbd9325b` 2764 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-linux-arm.tar.gz) | `947b0d9aeeef08961c0582b4c3c94b7ae1016d20b0c9f50af5fe760b3573f17497059511bcb57ac971a5bdadeb5c77dfd639d5745042ecc67541dd702ee7c657` 2765 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-linux-arm64.tar.gz) | `aff0258a223f5061552d340cda36872e3cd7017368117bbb14dc0f8a3a4db8c715c11743bedd72189cd43082aa9ac1ced64a6337c2f174bdcbeef094b47e76b0` 2766 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-linux-ppc64le.tar.gz) | `3eabecd62290ae8d876ae45333777b2c9959e39461197dbe90e6ba07d0a4c50328cbdf44e77d2bd626e435ffc69593d0e8b807b36601c19dd1a1ef17e6810b4f` 2767 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-linux-s390x.tar.gz) | `6651b2d95d0a8dd748c33c9e8018ab606b4061956cc2b6775bd0b008b04ea33df27be819ce6c391ceb2191b53acbbc088d602ed2d86bdd7a3a3fc1c8f876798a` 2768 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-windows-386.tar.gz) | `4b6c11b7a318e5fcac19144f6ab1638126c299e08c7b908495591674abcf4c7dd16f63c74c7d901beff24006150d2a31e0f75e28a9e14d6d0d88a09dafb014f0` 2769 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-client-windows-amd64.tar.gz) | `760ae08da6045ae7089fb27a9324e77bed907662659364857e1a8d103d19ba50e80544d8c21a086738b15baebfd9a5fa78d63638eff7bbe725436c054ba649cc` 2770 2771 ### Server Binaries 2772 2773 filename | sha512 hash 2774 -------- | ----------- 2775 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-server-linux-amd64.tar.gz) | `69db41f3d79aa0581c36a3736ab8dc96c92127b82d3cf25c5effc675758fe713ca7aa7e5b414914f1bc73187c6cee5f76d76b74a2ee1c0e7fa61557328f1b8ef` 2776 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-server-linux-arm.tar.gz) | `ca302f53ee91ab4feb697bb34d360d0872a7abea59c5f28cceefe9237a914c77d68722b85743998ab12bf8e42005e63a1d1a441859c2426c1a8d745dd33f4276` 2777 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-server-linux-arm64.tar.gz) | `79ab1f0a542ce576ea6d81cd2a7c068da6674177b72f1b5f5e3ca47edfdb228f533683a073857b6bc53225a230d15d3ba4b0cb9b6d5d78a309aa6e24c2f6c500` 2778 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-server-linux-ppc64le.tar.gz) | `fbe5b45326f1d03bcdd9ffd46ab454917d79f629ba23dae9d667d0c7741bc2f5db2960bf3c989bb75c19c9dc1609dacbb8a6dc9a440e5b192648e70db7f68721` 2779 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-server-linux-s390x.tar.gz) | `eb13ac306793679a3a489136bb7eb6588472688b2bb2aa0e54e61647d8c9da6d3589c19e7ac434c24defa78cb65f7b72593eedec1e7431c7ecae872298efc4de` 2780 2781 ### Node Binaries 2782 2783 filename | sha512 hash 2784 -------- | ----------- 2785 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-node-linux-amd64.tar.gz) | `a4bde88f3e0f6233d04f04d380d5f612cd3c574bd66b9f3ee531fa76e3e0f1c6597edbc9fa61251a377e8230bce0ce6dc1cf57fd19080bb7d13f14a391b27fe8` 2786 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-node-linux-arm.tar.gz) | `7d72aa8c1d883b9f047e5b98dbb662bdfd314f9c06af4213068381ffaac116e68d1aad76327ead7a4fd97976ea72277cebcf765c56b265334cb3a02c83972ec1` 2787 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-node-linux-arm64.tar.gz) | `c9380bb59ba26dcfe1ab52b5cb02e2d920313defda09ec7d19ccbc18f54def4b57cf941ac8a397392beb5836fdc12bc9600d4055f2cfd1319896cfc9631cab10` 2788 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-node-linux-ppc64le.tar.gz) | `7bcd79b368a62c24465fce7dcb024bb629eae034e09fb522fb43bb5798478ca2660a3ccc596b424325c6f69e675468900f3b41f3924e7ff453e3db40150b3c16` 2789 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-node-linux-s390x.tar.gz) | `9bda9dd24ee5ca65aaefece4213b46ef57cde4904542d94e6147542e42766f8b80fe24d99a6b8711bd7dbe00c415169a9f258f433c5f5345c2e17c2bb82f2670` 2790 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.2/kubernetes-node-windows-amd64.tar.gz) | `d5906f229d2d8e99bdb37e7d155d54560b82ea28ce881c5a0cde8f8d20bff8fd2e82ea4b289ae3e58616d3ec8c23ac9b473cb714892a377feb87ecbce156147d` 2791 2792 ## Changelog since v1.16.0-alpha.1 2793 2794 ### Action Required 2795 2796 * Revert "scheduler.alpha.kubernetes.io/critical-pod annotation is removed. Pod priority (spec.priorityClassName) should be used instead to mark pods as critical. Action required!" ([#80277](https://github.com/kubernetes/kubernetes/pull/80277), [@draveness](https://github.com/draveness)) 2797 * ACTION REQUIRED: container images tar files for 'amd64' will now contain the architecture in the RepoTags manifest.json section. ([#80266](https://github.com/kubernetes/kubernetes/pull/80266), [@javier-b-perez](https://github.com/javier-b-perez)) 2798 * If you are using docker manifests there are not visible changes. 2799 2800 ### Other notable changes 2801 2802 * Use HTTPS as etcd-apiserver protocol when mTLS between etcd and kube-apiserver on master is enabled, change etcd metrics/health port to 2382. ([#77561](https://github.com/kubernetes/kubernetes/pull/77561), [@wenjiaswe](https://github.com/wenjiaswe)) 2803 * kubelet: change node-lease-renew-interval to 0.25 of lease-renew-duration ([#80429](https://github.com/kubernetes/kubernetes/pull/80429), [@gaorong](https://github.com/gaorong)) 2804 * Fix error handling and potential go null pointer exception in kubeadm upgrade diff ([#80648](https://github.com/kubernetes/kubernetes/pull/80648), [@odinuge](https://github.com/odinuge)) 2805 * New flag --endpoint-updates-batch-period in kube-controller-manager can be used to reduce number of endpoints updates generated by pod changes. ([#80509](https://github.com/kubernetes/kubernetes/pull/80509), [@mborsz](https://github.com/mborsz)) 2806 * kubeadm: produce errors if they occur when resetting cluster status for a control-plane node ([#80573](https://github.com/kubernetes/kubernetes/pull/80573), [@bart0sh](https://github.com/bart0sh)) 2807 * When a load balancer type service is created in a k8s cluster that is backed by Azure Standard Load Balancer, the corresponding load balancer rule added in the Azure Standard Load Balancer would now have the "EnableTcpReset" property set to true. ([#80624](https://github.com/kubernetes/kubernetes/pull/80624), [@xuto2](https://github.com/xuto2)) 2808 * Update portworx plugin dependency on libopenstorage/openstorage to v1.0.0 ([#80495](https://github.com/kubernetes/kubernetes/pull/80495), [@adityadani](https://github.com/adityadani)) 2809 * Fixed detachment of deleted volumes on OpenStack / Cinder. ([#80518](https://github.com/kubernetes/kubernetes/pull/80518), [@jsafrane](https://github.com/jsafrane)) 2810 * when PodInfoOnMount is enabled for a CSI driver, the new csi.storage.k8s.io/ephemeral parameter in the volume context allows a driver's NodePublishVolume implementation to determine on a case-by-case basis whether the volume is ephemeral or a normal persistent volume ([#79983](https://github.com/kubernetes/kubernetes/pull/79983), [@pohly](https://github.com/pohly)) 2811 * Update gogo/protobuf to serialize backward, as to get better performance on deep objects. ([#77355](https://github.com/kubernetes/kubernetes/pull/77355), [@apelisse](https://github.com/apelisse)) 2812 * Remove GetReference() and GetPartialReference() function from pkg/api/ref, as the same function exists also in staging/src/k8s.io/client-go/tools/ref ([#80361](https://github.com/kubernetes/kubernetes/pull/80361), [@wojtek-t](https://github.com/wojtek-t)) 2813 * Fixed a bug in the CSI metrics that does not return not supported error when a CSI driver does not support metrics. ([#79851](https://github.com/kubernetes/kubernetes/pull/79851), [@jparklab](https://github.com/jparklab)) 2814 * Fixed a bug in kube-addon-manager's leader election logic that made all replicas active. ([#80575](https://github.com/kubernetes/kubernetes/pull/80575), [@mborsz](https://github.com/mborsz)) 2815 * Kibana has been slightly revamped/improved in the latest version ([#80421](https://github.com/kubernetes/kubernetes/pull/80421), [@lostick](https://github.com/lostick)) 2816 * kubeadm: fixed ignoring errors when pulling control plane images ([#80529](https://github.com/kubernetes/kubernetes/pull/80529), [@bart0sh](https://github.com/bart0sh)) 2817 * CRDs under k8s.io and kubernetes.io must have the "api-approved.kubernetes.io" set to either `unapproved.*` or a link to the pull request approving the schema. See https://github.com/kubernetes/enhancements/pull/1111 for more details. ([#79992](https://github.com/kubernetes/kubernetes/pull/79992), [@deads2k](https://github.com/deads2k)) 2818 * Reduce kube-proxy cpu usage in IPVS mode when a large number of nodePort services exist. ([#79444](https://github.com/kubernetes/kubernetes/pull/79444), [@cezarsa](https://github.com/cezarsa)) 2819 * Add CSI Migration Shim for VerifyVolumesAreAttached and BulkVolumeVerify ([#80443](https://github.com/kubernetes/kubernetes/pull/80443), [@davidz627](https://github.com/davidz627)) 2820 * Fix a bug that causes DaemonSet rolling update hang when there exist failed pods. ([#78170](https://github.com/kubernetes/kubernetes/pull/78170), [@DaiHao](https://github.com/DaiHao)) 2821 * Fix retry issues when the nodes are under deleting on Azure ([#80419](https://github.com/kubernetes/kubernetes/pull/80419), [@feiskyer](https://github.com/feiskyer)) 2822 * Add support for AWS EBS on windows ([#79552](https://github.com/kubernetes/kubernetes/pull/79552), [@wongma7](https://github.com/wongma7)) 2823 * Passing an invalid policy name in the `--cpu-manager-policy` flag will now cause the kubelet to fail instead of simply ignoring the flag and running the `cpumanager`s default policy instead. ([#80294](https://github.com/kubernetes/kubernetes/pull/80294), [@klueska](https://github.com/klueska)) 2824 * Add Filter extension point to the scheduling framework. ([#78477](https://github.com/kubernetes/kubernetes/pull/78477), [@YoubingLi](https://github.com/YoubingLi)) 2825 * cpuUsageNanoCores is now reported in the Kubelet summary API on Windows nodes ([#80176](https://github.com/kubernetes/kubernetes/pull/80176), [@liyanhui1228](https://github.com/liyanhui1228)) 2826 * `[]TopologySpreadConstraint` is introduced into PodSpec to support the "Even Pods Spread" alpha feature. ([#77327](https://github.com/kubernetes/kubernetes/pull/77327), [@Huang-Wei](https://github.com/Huang-Wei)) 2827 * kubeadm: fall back to client version in case of certain HTTP errors ([#80024](https://github.com/kubernetes/kubernetes/pull/80024), [@RainbowMango](https://github.com/RainbowMango)) 2828 * NFS Drivers are now enabled to collect metrics, StatFS metrics provider is used to collect the metrics. ([#75805](https://github.com/kubernetes/kubernetes/pull/75805) , [@brahmaroutu](https://github.com/brahmaroutu)) ([#75805](https://github.com/kubernetes/kubernetes/pull/75805), [@brahmaroutu](https://github.com/brahmaroutu)) 2829 * make node lease renew interval more heuristic based on node-status-update-frequency in kubelet ([#80173](https://github.com/kubernetes/kubernetes/pull/80173), [@gaorong](https://github.com/gaorong)) 2830 * Introduction of the pod overhead feature to the scheduler. This functionality is alpha-level as of ([#78319](https://github.com/kubernetes/kubernetes/pull/78319), [@egernst](https://github.com/egernst)) 2831 * Kubernetes v1.16, and is only honored by servers that enable the PodOverhead feature.gate. 2832 * N/A ([#80260](https://github.com/kubernetes/kubernetes/pull/80260), [@khenidak](https://github.com/khenidak)) 2833 * Add v1.Container.SecurityContext.WindowsOptions.RunAsUserName to the pod spec ([#79489](https://github.com/kubernetes/kubernetes/pull/79489), [@bclau](https://github.com/bclau)) 2834 * Pass-through volume MountOptions to global mount (NodeStageVolume) on the node for CSI ([#80191](https://github.com/kubernetes/kubernetes/pull/80191), [@davidz627](https://github.com/davidz627)) 2835 * Add Score extension point to the scheduling framework. ([#79109](https://github.com/kubernetes/kubernetes/pull/79109), [@ahg-g](https://github.com/ahg-g)) 2836 2837 2838 2839 # v1.16.0-alpha.1 2840 2841 [Documentation](https://docs.k8s.io) 2842 2843 ## Downloads for v1.16.0-alpha.1 2844 2845 2846 filename | sha512 hash 2847 -------- | ----------- 2848 [kubernetes.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes.tar.gz) | `4834c52267414000fa93c0626bded5a969cf65d3d4681c20e5ae2c5f62002a51dfb8ee869484f141b147990915ba57be96108227f86c4e9f571b4b25e7ed0773` 2849 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-src.tar.gz) | `9329d51f5c73f830f3c895c2601bc78e51d2d412b928c9dae902e9ba8d46338f246a79329a27e4248ec81410ff103510ba9b605bb03e08a48414b2935d2c164b` 2850 2851 ### Client Binaries 2852 2853 filename | sha512 hash 2854 -------- | ----------- 2855 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-darwin-386.tar.gz) | `3cedffb92a0fca4f0b2d41f8b09baa59dff58df96446e8eece4e1b81022d9fdda8da41b5f73a3468435474721f03cffc6e7beabb25216b089a991b68366c73bc` 2856 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-darwin-amd64.tar.gz) | `14de6bb296b4d022f50778b160c98db3508c9c7230946e2af4eb2a1d662d45b86690e9e04bf3e592ec094e12bed1f2bb74cd59d769a0eaac3c81d9b80e0a79c8` 2857 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-linux-386.tar.gz) | `8b2b9fa55890895239b99fabb866babe50aca599591db1ecf9429e49925ae478b7c813b9d7704a20f41f2d50947c3b3deecb594544f1f3eae6c4e97ae9bb9b70` 2858 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-linux-amd64.tar.gz) | `e927ac7b314777267b95e0871dd70c352ec0fc967ba221cb6cba523fa6f18d9d193e4ce92a1f9fa669f9c961de0e34d69e770ef745199ed3693647dd0d692e57` 2859 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-linux-arm.tar.gz) | `4a230a6d34e2ffd7df40c5b726fbcbb7ef1373d81733bfb75685b2448ed181eb49ef27668fc33700f30de88e5bbdcc1e52649b9d31c7940760f48c6e6eb2f403` 2860 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-linux-arm64.tar.gz) | `87c8d7185df23b3496ceb74606558d895a64daf0c41185c833a233e29216131baac6e356a57bb78293ed9d0396966ecc3b00789f2b66af352dc286b101bcc69a` 2861 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-linux-ppc64le.tar.gz) | `16ea5efa2fc29bc7448a609a7118e7994e901ab26462aac52f03b4851d4c9d103ee12d2335360f8aa503ddbb2a71f3000f0fcb33597dd813df4f5ad5f4819fa9` 2862 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-linux-s390x.tar.gz) | `7390ad1682227a70550b20425fa5287fecf6a5d413493b03df3a7795614263e7883f30f3078bbb9fbd389d2a1dab073f8f401be89b82bd5861fa6b0aeda579eb` 2863 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-windows-386.tar.gz) | `88251896dfe38e59699b879f643704c0195e7a5af2cb00078886545f49364a2e3b497590781f135b80d60e256bad3a4ea197211f4f061c98dee096f0845e7a9b` 2864 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-client-windows-amd64.tar.gz) | `766b2a9bf097e45b2549536682cf25129110bd0562ab0df70e841ff8657dd7033119b0929e7a213454f90594b19b90fa57d89918cee33ceadba7d689449fe333` 2865 2866 ### Server Binaries 2867 2868 filename | sha512 hash 2869 -------- | ----------- 2870 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-server-linux-amd64.tar.gz) | `dfd5c2609990c9b9b94249c654931b240dc072f2cc303e1e1d6dec1fddfb0a9e127e3898421ace00ab1947a3ad2f87cfd1266fd0b6193ef00f942269388ef372` 2871 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-server-linux-arm.tar.gz) | `7704c2d3c57950f184322263ac2be1649a0d737d176e7fed1897031d0efb8375805b5f12c7cf9ba87ac06ad8a635d6e399382d99f3cbb418961a4f0901465f50` 2872 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-server-linux-arm64.tar.gz) | `fbbd87cc38cfb6429e3741bfd87ecec4b69b551df6fb7c121900ced4c1cd0bc77a317ca8abd41f71ffd7bc0b1c7144fecb22fa405d0b211b238df24d28599333` 2873 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-server-linux-ppc64le.tar.gz) | `cfed5b936eb2fe44df5d0c9c6484bee38ef370fb1258522e8c62fb6a526e9440c1dc768d8bf33403451ae00519cab1450444da854fd6c6a37665ce925c4e7d69` 2874 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-server-linux-s390x.tar.gz) | `317681141734347260ad9f918fa4b67e48751f5a7df64a848d2a83c79a4e9dba269c51804b09444463ba88a2c0efa1c307795cd8f06ed840964eb2c725a4ecc3` 2875 2876 ### Node Binaries 2877 2878 filename | sha512 hash 2879 -------- | ----------- 2880 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-node-linux-amd64.tar.gz) | `b3b1013453d35251b8fc4759f6ac26bdeb37f14a98697078535f7f902e8ebca581b5629bbb4493188a7e6077eb5afc61cf275f42bf4d9f503b70bfc58b9730b2` 2881 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-node-linux-arm.tar.gz) | `0bacc1791d260d2863ab768b48daf66f0f7f89eeee70e68dd515b05fc9d7f14b466382fe16fa84a103e0023324f681767489d9485560baf9eb80fe0e7ffab503` 2882 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-node-linux-arm64.tar.gz) | `73bd70cb9d27ce424828a95d715c16fd9dd22396dbe1dfe721eb0aea9e186ec46e6978956613b0978a8da3c22df39790739b038991c0192281881fce41d7c9f1` 2883 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-node-linux-ppc64le.tar.gz) | `a865f98838143dc7e1e12d1e258e5f5f2855fcf6e88488fb164ad62cf886d8e2a47fdf186ad6b55172f73826ae19da9b2642b9a0df0fa08f9351a66aeef3cf17` 2884 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-node-linux-s390x.tar.gz) | `d2f9f746ed0fe00be982a847a3ae1b6a698d5c506be1d3171156902140fec64642ec6d99aa68de08bdc7d65c9a35ac2c36bda53c4db873cb8e7edc419a4ab958` 2885 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.16.0-alpha.1/kubernetes-node-windows-amd64.tar.gz) | `37f48a6d8174f38668bc41c81222615942bfe07e01f319bdfed409f83a3de3773dceb09fd86330018bb05f830e165e7bd85b3d23d26a50227895e4ec07f8ab98` 2886 2887 ## Changelog since v1.15.0 2888 2889 ### Action Required 2890 2891 * Migrate scheduler to use v1beta1 Event API. action required: any tool targeting scheduler events needs to use v1beta1 Event API ([#78447](https://github.com/kubernetes/kubernetes/pull/78447), [@yastij](https://github.com/yastij)) 2892 * scheduler.alpha.kubernetes.io/critical-pod annotation is removed. Pod priority (spec.priorityClassName) should be used instead to mark pods as critical. Action required! ([#79554](https://github.com/kubernetes/kubernetes/pull/79554), [@draveness](https://github.com/draveness)) 2893 * hyperkube: the `--make-symlinks` flag, deprecated in v1.14, has been removed. ([#80017](https://github.com/kubernetes/kubernetes/pull/80017), [@Pothulapati](https://github.com/Pothulapati)) 2894 * Node labels `beta.kubernetes.io/metadata-proxy-ready`, `beta.kubernetes.io/metadata-proxy-ready` and `beta.kubernetes.io/kube-proxy-ds-ready` are no longer added on new nodes. ([#79305](https://github.com/kubernetes/kubernetes/pull/79305), [@paivagustavo](https://github.com/paivagustavo)) 2895 * ip-mask-agent addon starts to use the label `node.kubernetes.io/masq-agent-ds-ready` instead of `beta.kubernetes.io/masq-agent-ds-ready` as its node selector. 2896 * kube-proxy addon starts to use the label `node.kubernetes.io/kube-proxy-ds-ready` instead of `beta.kubernetes.io/kube-proxy-ds-ready` as its node selector. 2897 * metadata-proxy addon starts to use the label `cloud.google.com/metadata-proxy-ready` instead of `beta.kubernetes.io/metadata-proxy-ready` as its node selector. 2898 * Kubelet removes the ability to set `kubernetes.io` or `k8s.io` labels via --node-labels other than the [specifically allowed labels/prefixes](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/279-limit-node-access/README.md#proposal). 2899 * The following APIs are no longer served by default: ([#70672](https://github.com/kubernetes/kubernetes/pull/70672), [@liggitt](https://github.com/liggitt)) 2900 * All resources under `apps/v1beta1` and `apps/v1beta2` - use `apps/v1` instead 2901 * `daemonsets`, `deployments`, `replicasets` resources under `extensions/v1beta1` - use `apps/v1` instead 2902 * `networkpolicies` resources under `extensions/v1beta1` - use `networking.k8s.io/v1` instead 2903 * `podsecuritypolicies` resources under `extensions/v1beta1` - use `policy/v1beta1` instead 2904 * Serving these resources can be temporarily re-enabled using the `--runtime-config` apiserver flag. 2905 * `apps/v1beta1=true` 2906 * `apps/v1beta2=true` 2907 * `extensions/v1beta1/daemonsets=true,extensions/v1beta1/deployments=true,extensions/v1beta1/replicasets=true,extensions/v1beta1/networkpolicies=true,extensions/v1beta1/podsecuritypolicies=true` 2908 * The ability to serve these resources will be completely removed in v1.18. 2909 * ACTION REQUIRED: Removed deprecated flag `--resource-container` from kube-proxy. ([#78294](https://github.com/kubernetes/kubernetes/pull/78294), [@vllry](https://github.com/vllry)) 2910 * The deprecated `--resource-container` flag has been removed from kube-proxy, and specifying it will now cause an error. The behavior is now as if you specified `--resource-container=""`. If you previously specified a non-empty `--resource-container`, you can no longer do so as of kubernetes 1.16. 2911 2912 ### Other notable changes 2913 2914 * When HPAScaleToZero feature gate is enabled HPA supports scaling to zero pods based on object or external metrics. HPA remains active as long as at least one metric value available. ([#74526](https://github.com/kubernetes/kubernetes/pull/74526), [@DXist](https://github.com/DXist)) 2915 * To downgrade the cluster to version that does not support scale-to-zero feature: 2916 * 1. make sure there are no hpa objects with minReplicas=0. Here is a oneliner to update it to 1: 2917 * $ kubectl get hpa --all-namespaces --no-headers=true | awk '{if($6==0) printf "kubectl patch hpa/%s --namespace=%s -p \"{\\"spec\\":{\\"minReplicas\\":1}}\" 2918 ", $2, $1 }' | sh 2919 * 2. disable HPAScaleToZero feature gate 2920 * Add support for writing out of tree custom scheduler plugins. ([#78162](https://github.com/kubernetes/kubernetes/pull/78162), [@hex108](https://github.com/hex108)) 2921 * Remove deprecated github.com/kardianos/osext dependency ([#80142](https://github.com/kubernetes/kubernetes/pull/80142), [@loqutus](https://github.com/loqutus)) 2922 * Add Bind extension point to the scheduling framework. ([#79313](https://github.com/kubernetes/kubernetes/pull/79313), [@chenchun](https://github.com/chenchun)) 2923 * On Windows systems, %USERPROFILE% is now preferred over %HOMEDRIVE%\%HOMEPATH% as the home folder if %HOMEDRIVE%\%HOMEPATH% does not contain a .kube* Add --kubernetes-version to "kubeadm init phase certs ca" and "kubeadm init phase kubeconfig" ([#80115](https://github.com/kubernetes/kubernetes/pull/80115), [@gyuho](https://github.com/gyuho)) 2924 * kubeadm ClusterConfiguration now supports featureGates: IPv6DualStack: true ([#80145](https://github.com/kubernetes/kubernetes/pull/80145), [@Arvinderpal](https://github.com/Arvinderpal)) 2925 * Fix a bug that ListOptions.AllowWatchBookmarks wasn't propagating correctly in kube-apiserver. ([#80157](https://github.com/kubernetes/kubernetes/pull/80157), [@wojtek-t](https://github.com/wojtek-t)) 2926 * Bugfix: csi plugin supporting raw block that does not need attach mounted failed ([#79920](https://github.com/kubernetes/kubernetes/pull/79920), [@cwdsuzhou](https://github.com/cwdsuzhou)) 2927 * Increase log level for graceful termination to v=5 ([#80100](https://github.com/kubernetes/kubernetes/pull/80100), [@andrewsykim](https://github.com/andrewsykim)) 2928 * kubeadm: support fetching configuration from the original cluster for 'upgrade diff' ([#80025](https://github.com/kubernetes/kubernetes/pull/80025), [@SataQiu](https://github.com/SataQiu)) 2929 * The sample-apiserver gains support for OpenAPI v2 spec serving at `/openapi/v2`. ([#79843](https://github.com/kubernetes/kubernetes/pull/79843), [@sttts](https://github.com/sttts)) 2930 * The `generate-internal-groups.sh` script in k8s.io/code-generator will generate OpenAPI definitions by default in `pkg/generated/openapi`. Additional API group dependencies can be added via `OPENAPI_EXTRA_PACKAGES=<group>/<version> <group2>/<version2>...`. 2931 * Cinder and ScaleIO volume providers have been deprecated and will be removed in a future release. ([#80099](https://github.com/kubernetes/kubernetes/pull/80099), [@dims](https://github.com/dims)) 2932 * kubelet's --containerized flag was deprecated in 1.14. This flag is removed in 1.16. ([#80043](https://github.com/kubernetes/kubernetes/pull/80043), [@dims](https://github.com/dims)) 2933 * Optimize EC2 DescribeInstances API calls in aws cloud provider library by querying instance ID instead of EC2 filters when possible ([#78140](https://github.com/kubernetes/kubernetes/pull/78140), [@zhan849](https://github.com/zhan849)) 2934 * etcd migration image no longer supports etcd2 version. ([#80037](https://github.com/kubernetes/kubernetes/pull/80037), [@dims](https://github.com/dims)) 2935 * Promote WatchBookmark feature to beta and enable it by default. ([#79786](https://github.com/kubernetes/kubernetes/pull/79786), [@wojtek-t](https://github.com/wojtek-t)) 2936 * With WatchBookmark feature, clients are able to request watch events with BOOKMARK type. Clients should not assume bookmarks are returned at any specific interval, nor may they assume the server will send any BOOKMARK event during a session. 2937 * update to use go 1.12.7 ([#79966](https://github.com/kubernetes/kubernetes/pull/79966), [@tao12345666333](https://github.com/tao12345666333)) 2938 * Add --shutdown-delay-duration to kube-apiserver in order to delay a graceful shutdown. `/healthz` will keep returning success during this time and requests are normally served, but `/readyz` will return faillure immediately. This delay can be used to allow the SDN to update iptables on all nodes and stop sending traffic. ([#74416](https://github.com/kubernetes/kubernetes/pull/74416), [@sttts](https://github.com/sttts)) 2939 * The `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration` APIs have been promoted to `admissionregistration.k8s.io/v1`: ([#79549](https://github.com/kubernetes/kubernetes/pull/79549), [@liggitt](https://github.com/liggitt)) 2940 * `failurePolicy` default changed from `Ignore` to `Fail` for v1 2941 * `matchPolicy` default changed from `Exact` to `Equivalent` for v1 2942 * `timeout` default changed from `30s` to `10s` for v1 2943 * `sideEffects` default value is removed and the field made required for v1 2944 * `admissionReviewVersions` default value is removed and the field made required for v1 (supported versions for AdmissionReview are `v1` and `v1beta1`) 2945 * The `name` field for specified webhooks must be unique for `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration` objects created via `admissionregistration.k8s.io/v1` 2946 * The `admissionregistration.k8s.io/v1beta1` versions of `MutatingWebhookConfiguration` and `ValidatingWebhookConfiguration` are deprecated and will no longer be served in v1.19. 2947 * The garbage collector and generic object quota controller have been updated to use the metadata client which improves memory ([#78742](https://github.com/kubernetes/kubernetes/pull/78742), [@smarterclayton](https://github.com/smarterclayton)) 2948 * and CPU usage of the Kube controller manager. 2949 * SubjectAccessReview requests sent for RBAC escalation, impersonation, and pod security policy authorization checks now populate the version attribute. ([#80007](https://github.com/kubernetes/kubernetes/pull/80007), [@liggitt](https://github.com/liggitt)) 2950 * na ([#79892](https://github.com/kubernetes/kubernetes/pull/79892), [@mikebrow](https://github.com/mikebrow)) 2951 * Use O_CLOEXEC to ensure file descriptors do not leak to subprocesses. ([#74691](https://github.com/kubernetes/kubernetes/pull/74691), [@cpuguy83](https://github.com/cpuguy83)) 2952 * The namespace controller has been updated to use the metadata client which improves memory ([#78744](https://github.com/kubernetes/kubernetes/pull/78744), [@smarterclayton](https://github.com/smarterclayton)) 2953 * and CPU usage of the Kube controller manager. 2954 * add `kubectl replace --raw` and `kubectl delete --raw` to have parity with create and get ([#79724](https://github.com/kubernetes/kubernetes/pull/79724), [@deads2k](https://github.com/deads2k)) 2955 * E2E tests no longer add command line flags directly to the command line, test suites that want that need to be updated if they don't use HandleFlags ([#75593](https://github.com/kubernetes/kubernetes/pull/75593), [@pohly](https://github.com/pohly)) 2956 * loading a -viper-config=e2e.yaml with suffix (introduced in 1.13) works again and now has a regression test 2957 * Kubernetes now supports transparent compression of API responses. Clients that send `Accept-Encoding: gzip` will now receive a GZIP compressed response body if the API call was larger than 128KB. Go clients automatically request gzip-encoding by default and should see reduced transfer times for very large API requests. Clients in other languages may need to make changes to benefit from compression. ([#77449](https://github.com/kubernetes/kubernetes/pull/77449), [@smarterclayton](https://github.com/smarterclayton)) 2958 * Resolves an issue serving aggregated APIs backed by services that respond to requests to `/` with non-2xx HTTP responses ([#79895](https://github.com/kubernetes/kubernetes/pull/79895), [@deads2k](https://github.com/deads2k)) 2959 * updated fluentd to 1.5.1, elasticsearchs & kibana to 7.1.1 ([#79014](https://github.com/kubernetes/kubernetes/pull/79014), [@monotek](https://github.com/monotek)) 2960 * kubeadm: implement support for concurrent add/remove of stacked etcd members ([#79677](https://github.com/kubernetes/kubernetes/pull/79677), [@neolit123](https://github.com/neolit123)) 2961 * Added a metric 'apiserver_watch_events_total' that can be used to understand the number of watch events in the system. ([#78732](https://github.com/kubernetes/kubernetes/pull/78732), [@mborsz](https://github.com/mborsz)) 2962 * KMS Providers will install a healthz check for the status of kms-pluign in kube-apiservers' encryption config. ([#78540](https://github.com/kubernetes/kubernetes/pull/78540), [@immutableT](https://github.com/immutableT)) 2963 * Fixes a bug in openapi published for custom resources using x-kubernetes-preserve-unknown-fields extensions, so that kubectl will allow sending unknown fields for that portion of the object. ([#79636](https://github.com/kubernetes/kubernetes/pull/79636), [@liggitt](https://github.com/liggitt)) 2964 * A new client `k8s.io/client-go/metadata.Client` has been added for accessing objects generically. This client makes it easier to retrieve only the metadata (the `metadata` sub-section) from resources on the cluster in an efficient manner for use cases that deal with objects generically, like the garbage collector, quota, or the namespace controller. The client asks the server to return a `meta.k8s.io/v1 PartialObjectMetadata` object for list, get, delete, watch, and patch operations on both normal APIs and custom resources which can be encoded in protobuf for additional work. If the server does not yet support this API the client will gracefully fall back to JSON and transform the response objects into PartialObjectMetadata. ([#77819](https://github.com/kubernetes/kubernetes/pull/77819), [@smarterclayton](https://github.com/smarterclayton)) 2965 * changes timeout value in csi plugin from 15s to 2min which fixes the timeout issue ([#79529](https://github.com/kubernetes/kubernetes/pull/79529), [@andyzhangx](https://github.com/andyzhangx)) 2966 * kubeadm: provide "--control-plane-endpoint" flag for `controlPlaneEndpoint` ([#79270](https://github.com/kubernetes/kubernetes/pull/79270), [@SataQiu](https://github.com/SataQiu)) 2967 * Fixes invalid "time stamp is the future" error when kubectl cp-ing a file ([#73982](https://github.com/kubernetes/kubernetes/pull/73982), [@tanshanshan](https://github.com/tanshanshan)) 2968 * Kubelet should now more reliably report the same primary node IP even if the set of node IPs reported by the CloudProvider changes. ([#79391](https://github.com/kubernetes/kubernetes/pull/79391), [@danwinship](https://github.com/danwinship)) 2969 * To configure controller manager to use ipv6dual stack: ([#73977](https://github.com/kubernetes/kubernetes/pull/73977), [@khenidak](https://github.com/khenidak)) 2970 * use --cluster-cidr="<cidr1>,<cidr2>". 2971 * Notes: 2972 2973 * 1. Only the first two cidrs are used (soft limits for Alpha, might be lifted later on). 2974 * 2. Only the "RangeAllocator" (default) is allowed as a value for --cidr-allocator-type . Cloud allocators are not compatible with ipv6dualstack 2975 * When using the conformance test image, a new environment variable E2E_USE_GO_RUNNER will cause the tests to be run with the new Golang-based test runner rather than the current bash wrapper. ([#79284](https://github.com/kubernetes/kubernetes/pull/79284), [@johnSchnake](https://github.com/johnSchnake)) 2976 * kubeadm: prevent PSP blocking of upgrade image prepull by using a non-root user ([#77792](https://github.com/kubernetes/kubernetes/pull/77792), [@neolit123](https://github.com/neolit123)) 2977 * kubelet now accepts a --cni-cache-dir option, which defaults to /var/lib/cni/cache, where CNI stores cache files. ([#78908](https://github.com/kubernetes/kubernetes/pull/78908), [@dcbw](https://github.com/dcbw)) 2978 * Update Azure API versions (containerregistry --> 2018-09-01, network --> 2018-08-01) ([#79583](https://github.com/kubernetes/kubernetes/pull/79583), [@justaugustus](https://github.com/justaugustus)) 2979 * Fix possible fd leak and closing of dirs in doSafeMakeDir ([#79534](https://github.com/kubernetes/kubernetes/pull/79534), [@odinuge](https://github.com/odinuge)) 2980 * kubeadm: fix the bug that "--cri-socket" flag does not work for `kubeadm reset` ([#79498](https://github.com/kubernetes/kubernetes/pull/79498), [@SataQiu](https://github.com/SataQiu)) 2981 * kubectl logs --selector will support --tail=-1. ([#74943](https://github.com/kubernetes/kubernetes/pull/74943), [@JishanXing](https://github.com/JishanXing)) 2982 * Introduce a new admission controller for RuntimeClass. Initially, RuntimeClass will be used to apply the pod overhead associated with a given RuntimeClass to the Pod.Spec if a corresponding RuntimeClassName is specified. ([#78484](https://github.com/kubernetes/kubernetes/pull/78484), [@egernst](https://github.com/egernst)) 2983 * PodOverhead is an alpha feature as of Kubernetes 1.16. 2984 * Fix kubelet errors in AArch64 with huge page sizes smaller than 1MiB ([#78495](https://github.com/kubernetes/kubernetes/pull/78495), [@odinuge](https://github.com/odinuge)) 2985 * The alpha `metadata.initializers` field, deprecated in 1.13, has been removed. ([#79504](https://github.com/kubernetes/kubernetes/pull/79504), [@yue9944882](https://github.com/yue9944882)) 2986 * Fix duplicate error messages in cli commands ([#79493](https://github.com/kubernetes/kubernetes/pull/79493), [@odinuge](https://github.com/odinuge)) 2987 * Default resourceGroup should be used when the value of annotation azure-load-balancer-resource-group is an empty string. ([#79514](https://github.com/kubernetes/kubernetes/pull/79514), [@feiskyer](https://github.com/feiskyer)) 2988 * Fixes output of `kubectl get --watch-only` when watching a single resource ([#79345](https://github.com/kubernetes/kubernetes/pull/79345), [@liggitt](https://github.com/liggitt)) 2989 * RateLimiter add a context-aware method, fix client-go request goruntine backlog in async timeout scene. ([#79375](https://github.com/kubernetes/kubernetes/pull/79375), [@answer1991](https://github.com/answer1991)) 2990 * Fix a bug where kubelet would not retry pod sandbox creation when the restart policy of the pod is Never ([#79451](https://github.com/kubernetes/kubernetes/pull/79451), [@yujuhong](https://github.com/yujuhong)) 2991 * Fix CRD validation error on 'items' field. ([#76124](https://github.com/kubernetes/kubernetes/pull/76124), [@tossmilestone](https://github.com/tossmilestone)) 2992 * The CRD handler now properly re-creates stale CR storage to reflect CRD update. ([#79114](https://github.com/kubernetes/kubernetes/pull/79114), [@roycaihw](https://github.com/roycaihw)) 2993 * Integrated volume limits for in-tree and CSI volumes into one scheduler predicate. ([#77595](https://github.com/kubernetes/kubernetes/pull/77595), [@bertinatto](https://github.com/bertinatto)) 2994 * Fix a bug in server printer that could cause kube-apiserver to panic. ([#79349](https://github.com/kubernetes/kubernetes/pull/79349), [@roycaihw](https://github.com/roycaihw)) 2995 * Mounts /home/kubernetes/bin/nvidia/vulkan/icd.d on the host to /etc/vulkan/icd.d inside containers requesting GPU. ([#78868](https://github.com/kubernetes/kubernetes/pull/78868), [@chardch](https://github.com/chardch)) 2996 * Remove CSIPersistentVolume feature gates ([#79309](https://github.com/kubernetes/kubernetes/pull/79309), [@draveness](https://github.com/draveness)) 2997 * Init container resource requests now impact pod QoS class ([#75223](https://github.com/kubernetes/kubernetes/pull/75223), [@sjenning](https://github.com/sjenning)) 2998 * Correct the maximum allowed insecure bind port for the kube-scheduler and kube-apiserver to 65535. ([#79346](https://github.com/kubernetes/kubernetes/pull/79346), [@ncdc](https://github.com/ncdc)) 2999 * Fix remove the etcd member from the cluster during a kubeadm reset. ([#79326](https://github.com/kubernetes/kubernetes/pull/79326), [@bradbeam](https://github.com/bradbeam)) 3000 * Remove KubeletPluginsWatcher feature gates ([#79310](https://github.com/kubernetes/kubernetes/pull/79310), [@draveness](https://github.com/draveness)) 3001 * Remove HugePages, VolumeScheduling, CustomPodDNS and PodReadinessGates feature flags ([#79307](https://github.com/kubernetes/kubernetes/pull/79307), [@draveness](https://github.com/draveness)) 3002 * The GA PodPriority feature gate is now on by default and cannot be disabled. The feature gate will be removed in v1.18. ([#79262](https://github.com/kubernetes/kubernetes/pull/79262), [@draveness](https://github.com/draveness)) 3003 * Remove pids cgroup controller requirement when related feature gates are disabled ([#79073](https://github.com/kubernetes/kubernetes/pull/79073), [@rafatio](https://github.com/rafatio)) 3004 * Add Bind extension point of the scheduling framework ([#78513](https://github.com/kubernetes/kubernetes/pull/78513), [@chenchun](https://github.com/chenchun)) 3005 * if targetPort is changed that will process by service controller ([#77712](https://github.com/kubernetes/kubernetes/pull/77712), [@Sn0rt](https://github.com/Sn0rt)) 3006 * update to use go 1.12.6 ([#78958](https://github.com/kubernetes/kubernetes/pull/78958), [@tao12345666333](https://github.com/tao12345666333)) 3007 * kubeadm: fix a potential panic if kubeadm discovers an invalid, existing kubeconfig file ([#79165](https://github.com/kubernetes/kubernetes/pull/79165), [@neolit123](https://github.com/neolit123)) 3008 * fix kubelet fail to delete orphaned pod directory when the kubelet's pods directory (default is "/var/lib/kubelet/pods") symbolically links to another disk device's directory ([#79094](https://github.com/kubernetes/kubernetes/pull/79094), [@gaorong](https://github.com/gaorong)) 3009 * Addition of Overhead field to the PodSpec and RuntimeClass types as part of the Pod Overhead KEP ([#76968](https://github.com/kubernetes/kubernetes/pull/76968), [@egernst](https://github.com/egernst)) 3010 * fix pod list return value of framework.WaitForPodsWithLabelRunningReady ([#78687](https://github.com/kubernetes/kubernetes/pull/78687), [@pohly](https://github.com/pohly)) 3011 * The behavior of the default handler for 404 requests fro the GCE Ingress load balancer is slightly modified in the sense that it now exports metrics using prometheus. The metrics exported include: ([#79106](https://github.com/kubernetes/kubernetes/pull/79106), [@vbannai](https://github.com/vbannai)) 3012 * - http_404_request_total (the number of 404 requests handled) 3013 * - http_404_request_duration_ms (the amount of time the server took to respond in ms) 3014 * Also includes percentile groupings. The directory for the default 404 handler includes instructions on how to enable prometheus for monitoring and setting alerts. 3015 * The kube-apiserver has improved behavior for both startup and shutdown sequences and also now exposes `/readyz` for readiness checking. Readyz includes all existing healthz checks but also adds a shutdown check. When a cluster admin initiates a shutdown, the kube-apiserver will try to process existing requests (for the duration of request timeout) before killing the apiserver process. ([#78458](https://github.com/kubernetes/kubernetes/pull/78458), [@logicalhan](https://github.com/logicalhan)) 3016 * The apiserver also now takes an optional flag "--maximum-startup-sequence-duration". This allows you to explicitly define an upper bound on the apiserver startup sequences before healthz begins to fail. By keeping the kubelet liveness initial delay short, this can enable quick kubelet recovery as soon as we have a boot sequence which has not completed in our expected time frame, despite lack of completion from longer boot sequences (like RBAC). Kube-apiserver behavior when the value of this flag is zero is backwards compatible (this is as the defaulted value of the flag). 3017 * fix: make azure disk URI as case insensitive ([#79020](https://github.com/kubernetes/kubernetes/pull/79020), [@andyzhangx](https://github.com/andyzhangx)) 3018 * Enable cadvisor ProcessMetrics collecting. ([#79002](https://github.com/kubernetes/kubernetes/pull/79002), [@jiayingz](https://github.com/jiayingz)) 3019 * Fixes a bug where `kubectl set config` hangs and uses 100% CPU on some invalid property names ([#79000](https://github.com/kubernetes/kubernetes/pull/79000), [@pswica](https://github.com/pswica)) 3020 * Fix a string comparison bug in IPVS graceful termination where UDP real servers are not deleted. ([#78999](https://github.com/kubernetes/kubernetes/pull/78999), [@andrewsykim](https://github.com/andrewsykim)) 3021 * Reflector watchHandler Warning log 'The resourceVersion for the provided watch is too old.' is now logged as Info. ([#78991](https://github.com/kubernetes/kubernetes/pull/78991), [@sallyom](https://github.com/sallyom)) 3022 * fix a bug that pods not be deleted from unmatched nodes by daemon controller ([#78974](https://github.com/kubernetes/kubernetes/pull/78974), [@DaiHao](https://github.com/DaiHao)) 3023 * Volume expansion is enabled in the default GCE storageclass ([#78672](https://github.com/kubernetes/kubernetes/pull/78672), [@msau42](https://github.com/msau42)) 3024 * kubeadm: use the service-cidr flag to pass the desired service CIDR to the kube-controller-manager via its service-cluster-ip-range flag. ([#78625](https://github.com/kubernetes/kubernetes/pull/78625), [@Arvinderpal](https://github.com/Arvinderpal)) 3025 * kubeadm: introduce deterministic ordering for the certificates generation in the phase command "kubeadm init phase certs" . ([#78556](https://github.com/kubernetes/kubernetes/pull/78556), [@neolit123](https://github.com/neolit123)) 3026 * Add Pre-filter extension point to the scheduling framework. ([#78005](https://github.com/kubernetes/kubernetes/pull/78005), [@ahg-g](https://github.com/ahg-g)) 3027 * fix pod stuck issue due to corrupt mnt point in flexvol plugin, call Unmount if PathExists returns any error ([#75234](https://github.com/kubernetes/kubernetes/pull/75234), [@andyzhangx](https://github.com/andyzhangx))