k8s.io/kubernetes@v1.31.0-alpha.0.0.20240520171757-56147500dadc/CHANGELOG/CHANGELOG-1.14.md (about) 1 <!-- BEGIN MUNGE: GENERATED_TOC --> 2 - [v1.14.10](#v11410) 3 - [Downloads for v1.14.10](#downloads-for-v11410) 4 - [Client Binaries](#client-binaries) 5 - [Server Binaries](#server-binaries) 6 - [Node Binaries](#node-binaries) 7 - [Changelog since v1.14.9](#changelog-since-v1149) 8 - [Other notable changes](#other-notable-changes) 9 - [v1.14.9](#v1149) 10 - [Downloads for v1.14.9](#downloads-for-v1149) 11 - [Client Binaries](#client-binaries-1) 12 - [Server Binaries](#server-binaries-1) 13 - [Node Binaries](#node-binaries-1) 14 - [Changelog since v1.14.8](#changelog-since-v1148) 15 - [Other notable changes](#other-notable-changes-1) 16 - [v1.14.8](#v1148) 17 - [Downloads for v1.14.8](#downloads-for-v1148) 18 - [Client Binaries](#client-binaries-2) 19 - [Server Binaries](#server-binaries-2) 20 - [Node Binaries](#node-binaries-2) 21 - [Changelog since v1.14.7](#changelog-since-v1147) 22 - [Other notable changes](#other-notable-changes-2) 23 - [v1.14.7](#v1147) 24 - [Downloads for v1.14.7](#downloads-for-v1147) 25 - [Client Binaries](#client-binaries-3) 26 - [Server Binaries](#server-binaries-3) 27 - [Node Binaries](#node-binaries-3) 28 - [Known Issues](#known-issues) 29 - [Changelog since v1.14.6](#changelog-since-v1146) 30 - [Other notable changes](#other-notable-changes-3) 31 - [v1.14.6](#v1146) 32 - [Downloads for v1.14.6](#downloads-for-v1146) 33 - [Client Binaries](#client-binaries-4) 34 - [Server Binaries](#server-binaries-4) 35 - [Node Binaries](#node-binaries-4) 36 - [Changelog since v1.14.5](#changelog-since-v1145) 37 - [Other notable changes](#other-notable-changes-4) 38 - [v1.14.5](#v1145) 39 - [Downloads for v1.14.5](#downloads-for-v1145) 40 - [Client Binaries](#client-binaries-5) 41 - [Server Binaries](#server-binaries-5) 42 - [Node Binaries](#node-binaries-5) 43 - [Changelog since v1.14.4](#changelog-since-v1144) 44 - [v1.14.4](#v1144) 45 - [Downloads for v1.14.4](#downloads-for-v1144) 46 - [Client Binaries](#client-binaries-6) 47 - [Server Binaries](#server-binaries-6) 48 - [Node Binaries](#node-binaries-6) 49 - [Changelog since v1.14.3](#changelog-since-v1143) 50 - [Other notable changes](#other-notable-changes-5) 51 - [v1.14.3](#v1143) 52 - [Downloads for v1.14.3](#downloads-for-v1143) 53 - [Client Binaries](#client-binaries-7) 54 - [Server Binaries](#server-binaries-7) 55 - [Node Binaries](#node-binaries-7) 56 - [Changelog since v1.14.2](#changelog-since-v1142) 57 - [Other notable changes](#other-notable-changes-6) 58 - [v1.14.2](#v1142) 59 - [Downloads for v1.14.2](#downloads-for-v1142) 60 - [Client Binaries](#client-binaries-8) 61 - [Server Binaries](#server-binaries-8) 62 - [Node Binaries](#node-binaries-8) 63 - [Changelog since v1.14.1](#changelog-since-v1141) 64 - [Other notable changes](#other-notable-changes-7) 65 - [v1.14.1](#v1141) 66 - [Downloads for v1.14.1](#downloads-for-v1141) 67 - [Client Binaries](#client-binaries-9) 68 - [Server Binaries](#server-binaries-9) 69 - [Node Binaries](#node-binaries-9) 70 - [Changelog since v1.14.0](#changelog-since-v1140) 71 - [Other notable changes](#other-notable-changes-8) 72 - [v1.14.0](#v1140) 73 - [Downloads for v1.14.0](#downloads-for-v1140) 74 - [Client Binaries](#client-binaries-10) 75 - [Server Binaries](#server-binaries-10) 76 - [Node Binaries](#node-binaries-10) 77 - [Kubernetes v1.14 Release Notes](#kubernetes-v114-release-notes) 78 - [1.14 What’s New](#114-whats-new) 79 - [Known Issues](#known-issues-1) 80 - [Urgent Upgrade Notes](#urgent-upgrade-notes) 81 - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade) 82 - [Deprecations](#deprecations) 83 - [Removed and deprecated metrics](#removed-and-deprecated-metrics) 84 - [Removed metrics](#removed-metrics) 85 - [Deprecated metrics](#deprecated-metrics) 86 - [Notable Features](#notable-features) 87 - [API Changes](#api-changes) 88 - [Detailed Bug Fixes And Changes](#detailed-bug-fixes-and-changes) 89 - [API Machinery](#api-machinery) 90 - [Apps](#apps) 91 - [Auth](#auth) 92 - [AWS](#aws) 93 - [Azure](#azure) 94 - [CLI](#cli) 95 - [Cloud Provider](#cloud-provider) 96 - [Cluster Lifecycle](#cluster-lifecycle) 97 - [GCP](#gcp) 98 - [Network](#network) 99 - [Node](#node) 100 - [Scheduling](#scheduling) 101 - [Storage](#storage) 102 - [Testing](#testing) 103 - [VMware](#vmware) 104 - [Windows](#windows) 105 - [External Dependencies](#external-dependencies) 106 - [v1.14.0-rc.1](#v1140-rc1) 107 - [Downloads for v1.14.0-rc.1](#downloads-for-v1140-rc1) 108 - [Client Binaries](#client-binaries-11) 109 - [Server Binaries](#server-binaries-11) 110 - [Node Binaries](#node-binaries-11) 111 - [Changelog since v1.14.0-beta.2](#changelog-since-v1140-beta2) 112 - [Action Required](#action-required) 113 - [Other notable changes](#other-notable-changes-9) 114 - [v1.14.0-beta.2](#v1140-beta2) 115 - [Downloads for v1.14.0-beta.2](#downloads-for-v1140-beta2) 116 - [Client Binaries](#client-binaries-12) 117 - [Server Binaries](#server-binaries-12) 118 - [Node Binaries](#node-binaries-12) 119 - [Changelog since v1.14.0-beta.1](#changelog-since-v1140-beta1) 120 - [Action Required](#action-required-1) 121 - [Other notable changes](#other-notable-changes-10) 122 - [v1.14.0-beta.1](#v1140-beta1) 123 - [Downloads for v1.14.0-beta.1](#downloads-for-v1140-beta1) 124 - [Client Binaries](#client-binaries-13) 125 - [Server Binaries](#server-binaries-13) 126 - [Node Binaries](#node-binaries-13) 127 - [Changelog since v1.14.0-alpha.3](#changelog-since-v1140-alpha3) 128 - [Action Required](#action-required-2) 129 - [Other notable changes](#other-notable-changes-11) 130 - [v1.14.0-alpha.3](#v1140-alpha3) 131 - [Downloads for v1.14.0-alpha.3](#downloads-for-v1140-alpha3) 132 - [Client Binaries](#client-binaries-14) 133 - [Server Binaries](#server-binaries-14) 134 - [Node Binaries](#node-binaries-14) 135 - [Changelog since v1.14.0-alpha.2](#changelog-since-v1140-alpha2) 136 - [Action Required](#action-required-3) 137 - [Other notable changes](#other-notable-changes-12) 138 - [v1.14.0-alpha.2](#v1140-alpha2) 139 - [Downloads for v1.14.0-alpha.2](#downloads-for-v1140-alpha2) 140 - [Client Binaries](#client-binaries-15) 141 - [Server Binaries](#server-binaries-15) 142 - [Node Binaries](#node-binaries-15) 143 - [Changelog since v1.14.0-alpha.1](#changelog-since-v1140-alpha1) 144 - [Action Required](#action-required-4) 145 - [Other notable changes](#other-notable-changes-13) 146 - [v1.14.0-alpha.1](#v1140-alpha1) 147 - [Downloads for v1.14.0-alpha.1](#downloads-for-v1140-alpha1) 148 - [Client Binaries](#client-binaries-16) 149 - [Server Binaries](#server-binaries-16) 150 - [Node Binaries](#node-binaries-16) 151 - [Changelog since v1.13.0](#changelog-since-v1130) 152 - [Action Required](#action-required-5) 153 - [Other notable changes](#other-notable-changes-14) 154 <!-- END MUNGE: GENERATED_TOC --> 155 156 <!-- NEW RELEASE NOTES ENTRY --> 157 158 159 # v1.14.10 160 161 [Documentation](https://docs.k8s.io) 162 163 ## Downloads for v1.14.10 164 165 166 filename | sha512 hash 167 -------- | ----------- 168 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes.tar.gz) | `b2b73d186769461236f94b7d1faa5d5806534bae5d9404f223f3e6aeaf1bc7a0c3bc505e2b8f3d34cec12d6657385927d82e67488f93ffde83c68239d563646d` 169 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-src.tar.gz) | `93f716ecbb29a5510eac8c6d075b842fc79cd02d979dd27a73c90065944854ff09d653ea68a734c9561049c8a16942460d84e6eba99156e9d888154e182da403` 170 171 ### Client Binaries 172 173 filename | sha512 hash 174 -------- | ----------- 175 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-darwin-386.tar.gz) | `baeb3f08d027766a1282a55fb5490e8c91b484d17952eb7a99623bdcb5a7b088cc7b6056302a27307d28a59727e7ef17d598dc6766f9201c8151684509b89d7a` 176 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-darwin-amd64.tar.gz) | `b6d3bd9589edf7282d3c6157fa271cdfdaca393470d7da0acecd8798884bcaa569ddb02d7a5d4374218504a2dbe1dd7d05bb77c280a6914f202bdd458f643cdc` 177 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-linux-386.tar.gz) | `9dc28d1a7e9ec20170d8980dc9da5c14fbce211b9117c248c92c64b9f0e954b179e35ee8b503d8a151a93a8b4572461fdae4437e5d5a2a451dbe1bd0ed0adea1` 178 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-linux-amd64.tar.gz) | `b60f3834a53642ca35c412c0aee0fb30a485005c2280d93bcb01added6012ac18c541c663b0a2a95129f233bc75a0787f0aca78a4e3546cd83e7708ce56a2eb7` 179 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-linux-arm.tar.gz) | `af29757e2dd82b031219ad3ea1c48a9e7bb03c15bd97d44dad93d0b30eb74dc8738a36ba8e2abda5516b6a15889f33eefdf951cc244f45e249616681997663ec` 180 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-linux-arm64.tar.gz) | `63ba22e2b2d37da43250398f15f3a61b94d0cef72947ddca8c81a7ee74d67ed04caa01107fc10e480a607bc51ab9a3d3aa842796397d57bd258e24308c0a46f7` 181 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-linux-ppc64le.tar.gz) | `c6ba8cf78bb2c2f1610d2585c297d4fbac345eb0c34407d63ef7eca9c1ca21cdf11c8a19f242ffa0db4a96cfb7af9eddcb7033b965cd85864767ee4124594337` 182 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-linux-s390x.tar.gz) | `671b88174e5c551da3e272381300cf23d512af2c8c52fae710f57ed1d8fa9b165855874e4630f56afddb6511973da01582a142d51efe7bbcdcc25dcdc56c57d1` 183 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-windows-386.tar.gz) | `8c70038716716e289aa05deda4e2cb723d82b0d9537e7dcebcd76bd40850ebb1f4ed15bca7e36250fb4548e55f08136d55c8ee0bb73e6d266bb032bc05877568` 184 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-client-windows-amd64.tar.gz) | `5cbe78764a3bd6e7c1ab8654027d73754fbcd5728819a65133b4fa355757ed9e4c70d90fc38013965dba2e520eae0758cfd1a8d7d2df4b5895f3d56cecc07b95` 185 186 ### Server Binaries 187 188 filename | sha512 hash 189 -------- | ----------- 190 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-server-linux-amd64.tar.gz) | `b66b3c1e34f56891c58d0ac4097f4c83be7422ef7201fde015cffcd1ec214e5fcb16788c04bf61bb88950e4824e1f7bdbc36fc4ac7142976ac6093041fbce507` 191 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-server-linux-arm.tar.gz) | `3e67d2013afe3a60d4e7df8363e887d79271ec59b5a19f0c32721e1bedd9dafae5a224271e1c89c16b34a7aa52513c25b8066db4453745f47208359df9270d6d` 192 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-server-linux-arm64.tar.gz) | `c3491e360340816f475f1b44165b81d4f531ce02a9b89cef286e03875c6bb672b1102a181ea42e6cad76b4f5de538b6185134a5be8b7f9055c91918c6216064c` 193 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-server-linux-ppc64le.tar.gz) | `70d831d7718340ff303afcdef7afbe09e832b7780e03985e6480fd747546610051a97e4d3e9300d7d802373ead05a3ea91c420b784fafdf0fa83c00b2589fdf8` 194 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-server-linux-s390x.tar.gz) | `86696bc9912616d645e0fa3facb3e9d0123a271657b2bc5a4802eb7e8a9e43c7fd40e9111729d7a1a8b8597506fd4c1d3909e6fa9e2a7ea553d03769c095cf2f` 195 196 ### Node Binaries 197 198 filename | sha512 hash 199 -------- | ----------- 200 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-node-linux-amd64.tar.gz) | `3f509a406e2c1c889ad447d37f9f34043d61f7e60f14dad2c6a94980283381695e99d058b6a783423d4c5e742d1d6c0c0e16b049dc2e49b6f203bce7c97da025` 201 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-node-linux-arm.tar.gz) | `d1196bfb3584d6bdd9e30907af7f3536c420eebea1d2db9b9506eba53668d4d739b88db8187aea7444ca99651afe35e665c2de91c0ec83524a667eab66888529` 202 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-node-linux-arm64.tar.gz) | `954dd6b23c2ff14e6bc5003039093c80f48315e89eb73e9cff0fca13ff66becbab24c51c5b84f2e3af599d9e408a03b4bf26a38ace61567d70ec0d1fe43c1dcc` 203 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-node-linux-ppc64le.tar.gz) | `a1c491707b4974acfaad6d04767dffb04800181aa1205a91019f56188714692c76df3d9d1e9af19ff12da536e149e5bf13a01cdda96ccbbe052ead084bd3b16b` 204 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-node-linux-s390x.tar.gz) | `d70599ca75b6024bfd8514801873fd4262cf5c7d6e4524217180854945d82f0634a7eb947ea4df996dc4db25a74c26614b2861b12ae79e05883aaef9bc12eefc` 205 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.10/kubernetes-node-windows-amd64.tar.gz) | `3f8bfc75734c19cc52e6e46bb843cfb429cba61cd7bfa8eab8273d39aa7b13bde142ef1d9daffac092265cde368add1c10a21800992f645bce08eaf914910d45` 206 207 ## Changelog since v1.14.9 208 209 ### Other notable changes 210 211 * azure: update disk lock logic per vm during attach/detach to allow concurrent updates for different nodes. ([#85115](https://github.com/kubernetes/kubernetes/pull/85115), [@aramase](https://github.com/aramase)) 212 * fix vmss dirty cache issue in disk attach/detach on vmss node ([#85158](https://github.com/kubernetes/kubernetes/pull/85158), [@andyzhangx](https://github.com/andyzhangx)) 213 * fix race condition when attach/delete azure disk in same time ([#84917](https://github.com/kubernetes/kubernetes/pull/84917), [@andyzhangx](https://github.com/andyzhangx)) 214 * fix azure load balancer update dns label issue ([#85329](https://github.com/kubernetes/kubernetes/pull/85329), [@nilo19](https://github.com/nilo19)) 215 * Ensure health probes are created for local traffic policy UDP services on Azure ([#85328](https://github.com/kubernetes/kubernetes/pull/85328), [@nilo19](https://github.com/nilo19)) 216 217 218 219 # v1.14.9 220 221 [Documentation](https://docs.k8s.io) 222 223 ## Downloads for v1.14.9 224 225 226 filename | sha512 hash 227 -------- | ----------- 228 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes.tar.gz) | `bc2de7897543f2c78b5a6ff0cddf60de7dafdb5586e3353b8105cb0ce5324c583289b5476cdaed4449224eae721068e6cbe7d9a69895517c753935eb367adde0` 229 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-src.tar.gz) | `e869da8eb3c9e6929e8c10f3aa8cf0ea63279935967a88cbe42d643427d43c13b456c08be40e58039161d0703ac3e696e26469498b2abb552d4d8a43c9c4aaf4` 230 231 ### Client Binaries 232 233 filename | sha512 hash 234 -------- | ----------- 235 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-darwin-386.tar.gz) | `cf78d75ffe2e5d618bc707b5d06d243336f4d04c51d6dcca01e37c9b85decfa47bb00cb65e1547b94007917eb67f87c608f24d0e85a751cc19b266a9e4b1c61d` 236 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-darwin-amd64.tar.gz) | `bc974988da19fbc5a518059486d1e0d9acd83f1a680ed227814a6f0a22ec2fd0eabb0b6f8e1392a6b16e331d3759bcf03191f1c0190c53b31a91f2b2a9b9788d` 237 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-linux-386.tar.gz) | `ddf0184d9897bf5ea7aec45af78fd0fedd96fa3cbfcff140efd53215883d8e5baeb650450144abe6e61752de7ba1ce2ddb71a77ee58f8034368059cf5126e50e` 238 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-linux-amd64.tar.gz) | `f7931bc403584f009ba68d7bfdb145d0d097306da06acf192aef0608446b5b91c64d1d7ff99182d08932644e0d3f30ecd96c6776c41d935b1312ae579d2539fa` 239 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-linux-arm.tar.gz) | `f9872a45b9b4c96e24e6c3922a45f8fd13f9d1fac7b69a644489836b823262aaefeca2f4cf47d1ddbe46f092cb4827af3da30721989c08a62a0e83f1a22b2fb2` 240 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-linux-arm64.tar.gz) | `631bedb727aaea5df486e63252ba2fbc90b74af5465631ca6fb77180af2ab9a75c583f209cf04545c147f7db6f3a733e3d16c9c6cf72d71af6bc549d39f0a69d` 241 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-linux-ppc64le.tar.gz) | `e99c2810b36457a1061ba6cd703ce54407a2cd17ca53c0b413d8553ffdad49ab34d4f82cf69823dff682070b596e7b7bff775ae0d195fd1d47c3828197988bf4` 242 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-linux-s390x.tar.gz) | `1182cafc35632dfb651be66f8b4b630b1cca0642cc464236c3bca5d3b0bdd85091566f0b90b18d89d879b9105a6182846d8d2105655f2ba8ffe2c5ff5e2d0a71` 243 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-windows-386.tar.gz) | `01e69a9a7f8d4aa005ebef770eb4bacf199d4d55d650be5669afb66502d49ccff3dbe6a85e4bc128ae4cf87ca1d53c0eebc4b2f569179506844fa8265a1b89fa` 244 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-client-windows-amd64.tar.gz) | `4387511eaa8229ff4ea08e183a46dc4670d64da576c25bf776bd22db738e7ad8effadb61d3942de98842a623e3f1f8815013f6b867c36b55f792daa15d2302ce` 245 246 ### Server Binaries 247 248 filename | sha512 hash 249 -------- | ----------- 250 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-server-linux-amd64.tar.gz) | `2e3905868e8f7cfc4b1e5a7ab49711c2c6c929f0f750bf6d932ada023b87cacf992f0b7b7be1a87ce4a50abe04098299a8c35072f4b6fe8af4c5bc55620f4301` 251 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-server-linux-arm.tar.gz) | `9d699ca53692d4316ea3b72fbcf4c0c6e00956879a0d5e396219fe600f371b62905ad0f2b30aeeacc02414983c72d20a5a5f9d840b3069edda58d06b662c6b04` 252 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-server-linux-arm64.tar.gz) | `4ab4e5e18ca21f79540ff538ea4ae6dd65b6af76c5c3f1ed34662357ff3321dce72953f53ab0e330da819f3962aa2039eb7806723705a9127e9a3efbf73221ab` 253 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-server-linux-ppc64le.tar.gz) | `7b72f6de77936f3a44e060d0fc0e45604bc1614cf78eecb56ddfb8b56873057fcfc701f29ab3b5d78e5dd3a320daf9a69811ffafe933523a82ae8a5c23a08887` 254 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-server-linux-s390x.tar.gz) | `76cb6e81b1c14cb650084c133ec62b5298f8d95988160ad1402d8169490a785a7d682f548540ecd8032254de1f0ecc118d66c89cc90e35d13fce091ac1539f05` 255 256 ### Node Binaries 257 258 filename | sha512 hash 259 -------- | ----------- 260 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-node-linux-amd64.tar.gz) | `b2600ef317d10cc56e7dc965b249d29c2a458eecb3207b06fd134c9e2a6f3a815b0d7cf73d21cd6c83d6cfa89fdc085319aa0c23eb180211ee1496bf23bac07c` 261 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-node-linux-arm.tar.gz) | `ecd617226b5b30b5bdebf7d9e7729371d25d36fabe672b2354a3ad7b9fdb042e09e61834d0a005eefa3c8e67aab8c52faf5e36b47e203cd64c735e186ceaa188` 262 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-node-linux-arm64.tar.gz) | `d4c5553dacab15c378d1e744342321943d6f8e66891ea199c00e4129219c11fd528bdb19beb32c185560e57cf92f6f4af173c51868e58c751f2ed406efccf33f` 263 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-node-linux-ppc64le.tar.gz) | `bee9f3c240267fef015fd472cbe2e837204b484e9756cd7073257aebf07ece225de9b4a3317c2cafc212ef668f4e8a22d3c8e1a231117d49e07773d070d3bfde` 264 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-node-linux-s390x.tar.gz) | `79156467ebeed81211bb62e6becf378b07ec91bfe3ee36ec9d623b95b271ca49f34595b10f7c68419183cbddc1a7a414f10f1e34535f7cf33495d1d1b7e39cfe` 265 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.9/kubernetes-node-windows-amd64.tar.gz) | `9d31f669c5dc04647fe44e546a96dc4f1d0e5df66ae44ea6be14d664b16ee3720ec37df4dab132ed34c0c657f42598a72dfff791caffb66f41a7ccf5a7df89b3` 266 267 ## Changelog since v1.14.8 268 269 ### Other notable changes 270 271 * Fix kubelet metrics gathering on non-English Windows hosts ([#84156](https://github.com/kubernetes/kubernetes/pull/84156), [@wawa0210](https://github.com/wawa0210)) 272 * kube-apiserver: Fixed a regression accepting patch requests > 1MB ([#84963](https://github.com/kubernetes/kubernetes/pull/84963), [@liggitt](https://github.com/liggitt)) 273 * kube-apiserver: fixed a bug that could cause a goroutine leak if the apiserver encountered an encoding error serving a watch to a websocket watcher ([#84693](https://github.com/kubernetes/kubernetes/pull/84693), [@tedyu](https://github.com/tedyu)) 274 * azure: Add allow unsafe read from cache ([#83685](https://github.com/kubernetes/kubernetes/pull/83685), [@aramase](https://github.com/aramase)) 275 * Adds a metric apiserver_request_error_total to kube-apiserver. This metric tallies the number of request_errors encountered by verb, group, version, resource, subresource, scope, component, and code. ([#84780](https://github.com/kubernetes/kubernetes/pull/84780), [@RainbowMango](https://github.com/RainbowMango)) 276 * fix: make azure disk URI as case insensitive ([#79020](https://github.com/kubernetes/kubernetes/pull/79020), [@andyzhangx](https://github.com/andyzhangx)) 277 * Fixed binding of block PersistentVolumes / PersistentVolumeClaims when BlockVolume feature is off. ([#84049](https://github.com/kubernetes/kubernetes/pull/84049), [@jsafrane](https://github.com/jsafrane)) 278 * Fixed an issue with informers missing an `Added` event if a recently deleted object was immediately recreated at the same time the informer dropped a watch and relisted. ([#83911](https://github.com/kubernetes/kubernetes/pull/83911), [@matte21](https://github.com/matte21)) 279 * CSI detach timeout increased from 10 seconds to 2 minutes ([#84321](https://github.com/kubernetes/kubernetes/pull/84321), [@cduchesne](https://github.com/cduchesne)) 280 * Update to use go1.12.12 ([#84064](https://github.com/kubernetes/kubernetes/pull/84064), [@cblecker](https://github.com/cblecker)) 281 * Fix handling tombstones in pod-disruption-budged controller. ([#83951](https://github.com/kubernetes/kubernetes/pull/83951), [@zouyee](https://github.com/zouyee)) 282 283 284 285 # v1.14.8 286 287 [Documentation](https://docs.k8s.io) 288 289 ## Downloads for v1.14.8 290 291 292 filename | sha512 hash 293 -------- | ----------- 294 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes.tar.gz) | `1aeaa04a0dfb3656204589fb503a0da98d085514ef82d1da930cf093fb17a0e11887c52aaeb1351ec87eb80227510de3a667984e6eb4acfcc1d3b5c8fb3681eb` 295 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-src.tar.gz) | `1946bb8035d3cf6b40b6e58a8d886cb0fbd023e656af5a1c68047cc2eb3c28541ec4fb07d7420ff269ca83e40dad0579d14faf745838b4db210df52f4d4f2117` 296 297 ### Client Binaries 298 299 filename | sha512 hash 300 -------- | ----------- 301 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-darwin-386.tar.gz) | `429099a2fface3c82650ec76b2b422d39f18b4efbfd209431a3f60d218f3c1874eff90499d9ec95a35c41b0a7e82579a25c05ac1a49c6bb530cc3fbf5b70cf6e` 302 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-darwin-amd64.tar.gz) | `e51446c4557eebe65f9c36bd19bc23b72fc90c7a5e3bf3fb070d63ddc518ee5ab2e12163933da70b353ce0e4bdee016b24f8393c9cddee31445b2ccc8a0868a9` 303 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-linux-386.tar.gz) | `6efe1d19c0c3998c0af559f33468bccc026b7e98d4efda1566b1e8adb9a4bb15dfa3dbc392a4ca211e40e21a8cf0ae27ce222beb1c8949922a532a8116316325` 304 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-linux-amd64.tar.gz) | `fe1d0816e843c228fdb9da659d61b217f2af9c05a4206bd091b93579142256354312a1fc4173193a0cf4c7f55f871c50eafc34358ecc128a7e790cd0c125a4f9` 305 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-linux-arm.tar.gz) | `cef8f09eb298569582c4861ef9bace4dc9ff3302bae798756f01d42bfb1e933379a9c842942b9c442de7b3d19cdc48f31c4aab2c4b5f48a9227fc790b4584cbd` 306 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-linux-arm64.tar.gz) | `47c64414d87e68382652db2e5292fda0a86fdda6b9e3abab61f8647d07005c22692225535966838b5a61b3d48805978b2f599eea2446923cab60f8ffb04d2845` 307 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-linux-ppc64le.tar.gz) | `0c4c026912b4ecd27c7ee9c995cc16b688c59cabd01f4702adbb6cceecf12f0fef311808d14d8482bf255c6cc06859214fb50d48a827a7dbece6a40d369e661c` 308 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-linux-s390x.tar.gz) | `9c7aac67511ec12b9176e9067c1d713605bf65f91c5adbf40b82f6e3598fb0676d9504377d7e653fb4a7958d3b14cee54256f9f07b968906985794f4e3e8068d` 309 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-windows-386.tar.gz) | `579c0276d3755a6cad3806fc2c5fce80d19dcbc3301a6dd5a2bf3853265444f8e8daed8156c30f8c7c337e6251a3e003239aa12e627225e2adc11a9f6f474640` 310 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-client-windows-amd64.tar.gz) | `7a7a7f4d3543d922c9526b601293f486973137c748e668f048329a375943b26fa5721cc00781b11ea7c6b75e60fa85c88cd95fe311656e4c9f49ec14544ca6fc` 311 312 ### Server Binaries 313 314 filename | sha512 hash 315 -------- | ----------- 316 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-server-linux-amd64.tar.gz) | `9fbdf873ff96bb47172157022182219ecce92131f1cea8f470340f5237532d8388c56ee4c821e3504ffcad16ade6193a216ffede44c2133d1907c320a255aafa` 317 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-server-linux-arm.tar.gz) | `7e0fb35a4322ca960b20feab1e9265daff3823bb83245753fa4ff763b55d5a2e91426bccb58bb4166fb1ee45bf58cb6452b9611b63609bff17fd694ac0d9d382` 318 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-server-linux-arm64.tar.gz) | `d5f48ee89bb0cd4151c636c8f6f777c49af31bfe5aca6ee62ad1327bbcff5430338a342b8ed023c5b030a4ee1e13e84f284a353520619092814e5aa542d7f1bd` 319 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-server-linux-ppc64le.tar.gz) | `1c2f57ae61e3facb8ec5e9cb47f36173df49870c6c51d08a6eef414c5568cdcdf8fa4ecd70c78851454a9caf05a1232ff0908ad7cebbf846553f824f413dded5` 320 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-server-linux-s390x.tar.gz) | `4eb8b48ceca578eb88446391858911c229f04bb8948930b521b40406ccc191295e7808d1353be8f4554319df669eb6953db3035f15dcaf88a3fe76d444b720db` 321 322 ### Node Binaries 323 324 filename | sha512 hash 325 -------- | ----------- 326 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-node-linux-amd64.tar.gz) | `05d32dde64178f71e4f95bd0354d25536b298a017c968803c1a9c7e34a591b63beb478c43909e3113dd8039a62df02455f4e8fc58d4a5ed22ade4e04087e6aee` 327 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-node-linux-arm.tar.gz) | `ce80d5df529a08dc03c27e53f9dcd79a36159e2e9f1a44d475b8151b55e3f94ee47c74f9ca18c169a8a536fdce5aef0afb464d94a17a650410eb34bcfe3191ce` 328 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-node-linux-arm64.tar.gz) | `8905f2fc110439bf992239e09d0dca7cbba918ef2652ab9a1e511361ba3770433edbad0a13fb66b30c08530e4190f8a26c9e984f0a7126ee0c04262b77c8e623` 329 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-node-linux-ppc64le.tar.gz) | `ccf0dd819fdba4c8e3a32e16fb2d26088b8bce84946a885b5ff2ce56c4704d1d95a855bae8929388af4c714a83b87a77b1febf4d4fc255f2737fc1ce402235c5` 330 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-node-linux-s390x.tar.gz) | `c157679a832d0f76bde570d243bc9ddaf21663808a12f6ab4a3477118dea63aafeab67f0118460efd81e8770547f9ed0f6b0064221ecb72ddea424eba6198cc9` 331 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.8/kubernetes-node-windows-amd64.tar.gz) | `7c87c34736ead06b1823ce004edf27a24773d873d6dff594a587621f3480ff134c7d912fcddc1eb30d4a66538a2403e2ccb477920676c2cddc8a8bcf81e20d4b` 332 333 ## Changelog since v1.14.7 334 335 ### Other notable changes 336 337 * Fixes an issue in the AWS cloudprovider, where ELBs fail to register instances ([#82954](https://github.com/kubernetes/kubernetes/pull/82954), [@M00nF1sh](https://github.com/M00nF1sh)) 338 * Fixes a goroutine leak in kube-apiserver when a request times out. ([#83333](https://github.com/kubernetes/kubernetes/pull/83333), [@lavalamp](https://github.com/lavalamp)) 339 * Fix aggressive VM calls for Azure VMSS ([#83102](https://github.com/kubernetes/kubernetes/pull/83102), [@feiskyer](https://github.com/feiskyer)) 340 * Fixes a flaw (CVE-2019-11253) in json/yaml decoding where large or malformed documents could consume excessive server resources. Request bodies for normal API requests (create/delete/update/patch operations of regular resources) are now limited to 3MB. ([#83261](https://github.com/kubernetes/kubernetes/pull/83261), [@liggitt](https://github.com/liggitt)) 341 * Resolves bottleneck in internal API server communication that can cause increased goroutines and degrade API Server performance ([#80465](https://github.com/kubernetes/kubernetes/pull/80465), [@answer1991](https://github.com/answer1991)) 342 * Update to go 1.12.10 ([#83139](https://github.com/kubernetes/kubernetes/pull/83139), [@cblecker](https://github.com/cblecker)) 343 * fix: azure disk detach failure if node not exists ([#82640](https://github.com/kubernetes/kubernetes/pull/82640), [@andyzhangx](https://github.com/andyzhangx)) 344 345 346 347 # v1.14.7 348 349 [Documentation](https://docs.k8s.io) 350 351 ## Downloads for v1.14.7 352 353 354 filename | sha512 hash 355 -------- | ----------- 356 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes.tar.gz) | `c7ef389927f050fd365a5c4d31a6e4a0b0c8c910cec0a36547ed42338da9c11de528c98910ed9c971e1c7edb278812444f62f5e014d2253e163d7d3fd6b28ad9` 357 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-src.tar.gz) | `4d0f790d2b3dddb961874ec89a9de5547682662ea35f4d7c1fcf774548fb238b07d5ad90259137c0d84d4d3ad3776d228ce54443eabb9466317c0fe93acfa00d` 358 359 ### Client Binaries 360 361 filename | sha512 hash 362 -------- | ----------- 363 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-darwin-386.tar.gz) | `ffc9a90f252bdfe40562c63c2a607e832f883fb734731f22bcc436a7bcb6c58029204262d4b3a29e1c9de97e76e44117c38cdb025a83b223942d919d238df8a7` 364 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-darwin-amd64.tar.gz) | `04278365c8b3e75b3515ab8982756b613ff79ac76aaf06cfc684d801f00db7485b8090dae43a468f7deb67fc3adf27f52933c85a5246dd5ccaf0ec43791ea46a` 365 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-linux-386.tar.gz) | `12c63a07b0dfc73fa2d08016987baf844ff86076f0abb184b813463479c7ce454856ad66af807492dc136eda29e365be5ca3e9b80423284e120d3381715e5228` 366 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-linux-amd64.tar.gz) | `6d84db16061dd8d3e3eb49c94d2559d0c7cb4896f0ca3003259c733da4fd2de8de87619cff27249b0f8b29fb56f94e150b2dd2d64186e4195bda46e1e8048092` 367 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-linux-arm.tar.gz) | `7c5245c50d4b20d90a5e9726fd0dc6160644310008960476b04a998b8c557fcfdba0e3127ee17afc743decdab781d88c7efe7b38c0d19f55bd60a74c9e520389` 368 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-linux-arm64.tar.gz) | `68fc7335e6f0bc9a3c9f8b70da136d568606e8479d0568d928e04159164e15a118270ebdd39a32ff278f21870098dcf727331d3bf1b5151d9bac7d28e8f4092b` 369 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-linux-ppc64le.tar.gz) | `3347649520483c6a176d3d07ade878cc1949dce7d522b3c0b39822f93b8e1acd8d28c1445766901af71a445099af3b6e38e5794dbbed01aa51d24701f7fa97c9` 370 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-linux-s390x.tar.gz) | `a5dcf9fdf24aaa61c7224fab38018cf3f893a15337cab4dcad0b394bbb1e6a340912d0f643bbac391ff07a263b537df0920b6f7f4aa0057b85c437b372a2848f` 371 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-windows-386.tar.gz) | `d2ca52057ba30fb579ab9843abe68f98c15b6fb7bfcdc83e587108e814bca38cdb13f088c147bc53244606cbf4ad189abfa1be15ffbf740253974e7b200b525a` 372 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-client-windows-amd64.tar.gz) | `542e1dddb45a242730e103179637f1a9db8c79ec7e5bf37695e5e09e2ee8bb484c9b2180f23178801b4411df300a499d76c9e66fcb89e182be614a75baa26f37` 373 374 ### Server Binaries 375 376 filename | sha512 hash 377 -------- | ----------- 378 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-server-linux-amd64.tar.gz) | `108b4fa348c2504d066c15d9bdc3d418780858fca21ce8cb561c30f318b9be155be932333e661ced7cee5b6332847d7041346eb6218f6833a659840186a6c672` 379 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-server-linux-arm.tar.gz) | `c19e1af460d52d2220531dbf899973d9388c22deebf636dbfcb99425f2ac28c9949ff867abf4e070f924600a1ea56817e24804df4dbec37de0f8ae0a9ef0e4f1` 380 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-server-linux-arm64.tar.gz) | `f2deeeb924a8202d21479db7a4812ce052da9cc5bde9ca40000c4d1a26b45879a5924c8a863a3b1b54a8d7fb28c63598e57e3a8bf67d4c27666f71e0dab3384c` 381 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-server-linux-ppc64le.tar.gz) | `a548c9114f3e9cde8f07892e0c39843e5fad0f81a8b722a8c61b345d651d8fd3d51dc0ccdc9f790162a0a3958aa0b613cde42f0e4c882bc57b398007ba2feb98` 382 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-server-linux-s390x.tar.gz) | `d905534835fb61856ad5257cf9e9a2353720e8826291d03f036405acf80c769ad3ba6cf6d7d8f90e23bb189eb3491b6188fa122a41618a1fd6f141783ff3298c` 383 384 ### Node Binaries 385 386 filename | sha512 hash 387 -------- | ----------- 388 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-node-linux-amd64.tar.gz) | `d34df64b7593ee4fe01e85350968caf31bb6f0e8726af8966a978ea8f1b8436a5225e64ca75486d62ea0ad744ac4b8abe1d58e715aebfa561b27935080def858` 389 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-node-linux-arm.tar.gz) | `d2e570dcedcb36104272616b506802857bea96aef15f1b39425361965ecf3f6da5863a7d2acd2f8f254073ac5ba71e5c678f5554feabc80d7524a39b985c9b19` 390 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-node-linux-arm64.tar.gz) | `092aa8bdd4741eaf80aa7370d0e214779b984713a02467cb7d107201bbdf04d27ad7b0005302e477cbd009ba2c26a0ccbf33ab93efe3237466a8c9046603a013` 391 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-node-linux-ppc64le.tar.gz) | `89b6c5b9db9a0bd48b33daaf526a6367584b65013a181c8d500641d88d1c4d1e6e5295e671237636f0d095ad5aac8e9719405bf1b9c44a69d63b722de376f11d` 392 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-node-linux-s390x.tar.gz) | `bca2c439588fccb4f10f6bb5f2f06bdb9606ed9baf3101cd792b8b826c668905fddc82bd81b99204592f8b42a3e3ded9da9b96418a3ba20e41a1f67d20520cce` 393 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.7/kubernetes-node-windows-amd64.tar.gz) | `e67828b5998342f8095f102e9c91e6ad5c59dbae3566c5acc4a83bb042b3ddb8118f224a11a44bbcb4af8c34a852c973b52f4d871584f718f982629206b29153` 394 395 ## Known Issues 396 397 * There is a known issue [#82923](https://github.com/kubernetes/kubernetes/issues/82923) in the AWS cloudprovider, where ELBs fail to register instances. The fix has been merged [#82954](https://github.com/kubernetes/kubernetes/pull/82954) and will be included in v1.14.8. 398 399 ## Changelog since v1.14.6 400 401 ### Other notable changes 402 403 * When the number of jobs exceeds 500, cronjob should schedule without error. ([#77475](https://github.com/kubernetes/kubernetes/pull/77475), [@liucimin](https://github.com/liucimin)) 404 * Fix a bug in apiserver that could cause a valid update request to be rejected with a precondition check failure. ([#82303](https://github.com/kubernetes/kubernetes/pull/82303), [@roycaihw](https://github.com/roycaihw)) 405 * kubectl cp now safely allows unpacking of symlinks that may point outside the destination directory ([#82384](https://github.com/kubernetes/kubernetes/pull/82384), [@tallclair](https://github.com/tallclair)) 406 * Limit use of tags when calling EC2 API to prevent API throttling for very large clusters ([#76749](https://github.com/kubernetes/kubernetes/pull/76749), [@mcrute](https://github.com/mcrute)) 407 * fix azure disk naming matching issue due to case sensitive comparison ([#81720](https://github.com/kubernetes/kubernetes/pull/81720), [@andyzhangx](https://github.com/andyzhangx)) 408 * Fix `kubectl logs -f` for windows server containers. ([#81747](https://github.com/kubernetes/kubernetes/pull/81747), [@Random-Liu](https://github.com/Random-Liu)) 409 * Fix VMSS LoadBalancer backend pools so that the network won't be broken when instances are upgraded to latest model ([#81411](https://github.com/kubernetes/kubernetes/pull/81411), [@nilo19](https://github.com/nilo19)) 410 * fix: detach azure disk issue using dangling error ([#81266](https://github.com/kubernetes/kubernetes/pull/81266), [@andyzhangx](https://github.com/andyzhangx)) 411 * remove iSCSI volume storage cleartext secrets in logs ([#81215](https://github.com/kubernetes/kubernetes/pull/81215), [@zouyee](https://github.com/zouyee)) 412 * Fix a bug in server printer that could cause kube-apiserver to panic. ([#79349](https://github.com/kubernetes/kubernetes/pull/79349), [@roycaihw](https://github.com/roycaihw)) 413 * Fix a bug in the IPVS proxier where virtual servers are not cleaned up even though the corresponding Service object was deleted. ([#80942](https://github.com/kubernetes/kubernetes/pull/80942), [@gongguan](https://github.com/gongguan)) 414 415 # v1.14.6 416 417 [Documentation](https://docs.k8s.io) 418 419 ## Downloads for v1.14.6 420 421 422 filename | sha512 hash 423 -------- | ----------- 424 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes.tar.gz) | `efa84e3a0704ba093bd1d55f82510a5cb48aaf7b31068e3d14782739692efe60716d5a64528d5a35f0e089a9e6ff490676cd39630a3b68ca04b35aa210eedd5a` 425 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-src.tar.gz) | `24bde69a7b503b23a0ce168688f6cf0227ac515ae7e0edb9faef346805c36b159bfd81cec5779bd02ed7c7a7259369c8545a9affc0af42c16e6763c5f5fd669b` 426 427 ### Client Binaries 428 429 filename | sha512 hash 430 -------- | ----------- 431 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-darwin-386.tar.gz) | `587f7c1a0555b3b9bc91d2f392d2bbe7342c3f1a9502eb109667e4cc06888fb1ea74665b112dfd4b7bf3fa037f2c263bafaecb0ea443faf2697a83dce8e8f497` 432 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-darwin-amd64.tar.gz) | `afed84639adb98fed7087a238415dbbbb6643ab7403ecea11a08ea8b230312551a5770de1b42fd0a379a4d0ca96e934f57ebf7c0ad5d11f282b75e381ef27c04` 433 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-linux-386.tar.gz) | `a5769eaf1dc8ea87c9e74f9992ba1d8772a56e839da6c8dac77a2e90f0bc7da67370a0bfeb5ef4024ec604ac48edd4f07a1d58dce7025d087b01ac6fffdca325` 434 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-linux-amd64.tar.gz) | `59624a0d591ba210f8db5bfa1ce7e0e5b3f5c2774fea356f7e459db5d245a18d72ffc315f82b2f97d0e283c269eeb40fd6eca683cd76b35fb83cfd3bc87cc911` 435 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-linux-arm.tar.gz) | `d94de8e20c2ef224a8ec706ea65e321c83aefd4e7b64e99df1ac9bba0f8bc743c884bd462fa6243293141cb589eb802b9eae19bb965db18d7fbebfdf3791a8f2` 436 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-linux-arm64.tar.gz) | `8668c27593c10578fee89f609e0eb61ab3b092853ab4fcad83eff081aed4055893387073fe2a55669db623beb2ab7f1d5bd154ba309435bd36d29780b1737d60` 437 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-linux-ppc64le.tar.gz) | `dfa9c32dfa60093e2d9442ab036be7ff48a48c0305f88c4b4b958153473e1826f895fc98a92b21ef04505b0ddda0622f3cc47e2a8f40adc86fd74a486369b3e2` 438 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-linux-s390x.tar.gz) | `5d3d15f8aacfa61c94b7d2bf3b13855c0fc71441fd20e3cb43f8f9dc11925609e86b355b695a8bb189887ccdc98cf30ce47a7afddcc33ebb2bfd9b4e50039d97` 439 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-windows-386.tar.gz) | `1f6101dffa7de2345154e85bf78d87e85af4db23d3773182dbab2d382676e5977e477217225d0506ae4b0cd37ad3dc7f9bbb7f51af1f83a0a92d3c0bcd0f67d7` 440 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-client-windows-amd64.tar.gz) | `db9afa90d12cbc99b7b1d9c4c01744da06d11ac7c4c3eeef5033e895757a83de6cbb9ef6aa89c1bec14ed1cfb808f1ba37b845d5b9a3e530f25b9965200d91e4` 441 442 ### Server Binaries 443 444 filename | sha512 hash 445 -------- | ----------- 446 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-server-linux-amd64.tar.gz) | `8015cc64f10d25a0fa7890252084c1d256818f161e7dab8fe9a44cf9b74b0b20a1577ae21220b7f2e90745aaddb8fef4a2c51e5e9c389e3b616d21b2678209f6` 447 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-server-linux-arm.tar.gz) | `476d99a3c05a81480f7e4f197b113f3b1294921686a45184ab5342fcab796be28c42c444078cb78cdd9ef07dfcc874679c64256935b6012470dc3969fc0b70e6` 448 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-server-linux-arm64.tar.gz) | `524b65e0d41f72783aea50e01e6ccdca86467f867d7d51e934bce208229058c9df17d5be20491e272685ef034c7bcbffa353377e3340cc2ce394586ff4e82fd7` 449 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-server-linux-ppc64le.tar.gz) | `479b4017695c6aa245ba87a8828ad83e7dce92b73590b991af1427077eaab93db11209fc1b755aae639dfcaa95de328b20b6c216ebddee74da242fad29ca010e` 450 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-server-linux-s390x.tar.gz) | `5fa9d87803e2ddb488ff8fbf06df8d78d5c39d7c97d3d690ea33a7a0a5000b9095645bb160a222f5494b74c571a3bc678690f14a266e30d9bd5ab4a4148e4509` 451 452 ### Node Binaries 453 454 filename | sha512 hash 455 -------- | ----------- 456 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-node-linux-amd64.tar.gz) | `7987d3cea7a985aae0211cf03257501eba2218f0038152a4da2ee623f00feb6a26a46db04ffd619d3d7b9923331a477eed733e07eb9ce780eedb7c4084a04544` 457 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-node-linux-arm.tar.gz) | `907c019335e531e0316d3538e1f0ff0f827e6eb83d46cee5a64270b9826f7ea89468a8f399ea1922acfe2a78472ce6e1a5aca13fd1b52de695a67ee45d527dbf` 458 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-node-linux-arm64.tar.gz) | `3b1470381c806ed29482669cc36bfdec425e9f8f88211b75fa48dde4a29ef6b2e768736741d56f49127f73b17b82949371e0940d8a202efd1de99ebda0a04727` 459 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-node-linux-ppc64le.tar.gz) | `2988fa171f0f1bd73e9e6424d9b87aada1b77f0464551479709f51e52fa008805fff1a48edd1e81574bc33e3b0f822074f261a99561ffeed6a701072cdbdb7d6` 460 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-node-linux-s390x.tar.gz) | `2ab0fd57591ca42b9a43df110efdb3b7265acd6d94c3d3d62e09192a7374bb76ecbf89264d8bb7487fc6dd13be69f309510f21a2db44f51a8a1f377655d1a59a` 461 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.6/kubernetes-node-windows-amd64.tar.gz) | `23ab6aabea7eeb95e57a1a392984e9bdd0e7064373eb7b38f9f4bfc7c56e3163604a20a57fbb31924bc4b6e0b2c21899dce1f682b8c4b1308a3ba78211fce061` 462 463 ## Changelog since v1.14.5 464 465 ### Other notable changes 466 467 * update to use go 1.12.6 ([#78958](https://github.com/kubernetes/kubernetes/pull/78958), [@tao12345666333](https://github.com/tao12345666333)) 468 * update to use go 1.12.7 ([#79966](https://github.com/kubernetes/kubernetes/pull/79966), [@tao12345666333](https://github.com/tao12345666333)) 469 * Update to use go 1.12.8 ([#81390](https://github.com/kubernetes/kubernetes/pull/81390), [@cblecker](https://github.com/cblecker)) 470 * Update to use go 1.12.9 ([#81489](https://github.com/kubernetes/kubernetes/pull/81489), [@BenTheElder](https://github.com/BenTheElder)) 471 * Update golang/x/net dependency to bring in fixes for CVE-2019-9512, CVE-2019-9514 ([#81525](https://github.com/kubernetes/kubernetes/pull/81525), [@cblecker](https://github.com/cblecker)) 472 * cpuUsageNanoCores is now reported in the Kubelet summary API on Windows nodes ([#80176](https://github.com/kubernetes/kubernetes/pull/80176), [@liyanhui1228](https://github.com/liyanhui1228)) 473 * API: the metadata.selfLink field is deprecated in individual and list objects. It will no longer be returned starting in v1.20, and the field will be removed entirely in v1.21. ([#80978](https://github.com/kubernetes/kubernetes/pull/80978), [@wojtek-t](https://github.com/wojtek-t)) 474 * Fix Azure client requests stuck issues on http.StatusTooManyRequests (HTTP Code 429). ([#81279](https://github.com/kubernetes/kubernetes/pull/81279), [@feiskyer](https://github.com/feiskyer)) 475 * Update the GCE windows node image to include hot fixes since July. ([#81236](https://github.com/kubernetes/kubernetes/pull/81236), [@YangLu1031](https://github.com/YangLu1031)) 476 * Fix public IP not found issues for VMSS nodes ([#80703](https://github.com/kubernetes/kubernetes/pull/80703), [@feiskyer](https://github.com/feiskyer)) 477 * Fix a conformance test that was skipped when client had a newer micro version than server. ([#80598](https://github.com/kubernetes/kubernetes/pull/80598), [@smarterclayton](https://github.com/smarterclayton)) 478 * Fix error in `kubeadm join --discovery-file` when using discovery files with embedded credentials ([#80675](https://github.com/kubernetes/kubernetes/pull/80675), [@fabriziopandini](https://github.com/fabriziopandini)) 479 * make node lease renew interval more heuristic based on node-status-update-frequency in kubelet ([#80173](https://github.com/kubernetes/kubernetes/pull/80173), [@gaorong](https://github.com/gaorong)) 480 * Pass-through volume MountOptions to global mount (NodeStageVolume) on the node for CSI ([#80191](https://github.com/kubernetes/kubernetes/pull/80191), [@davidz627](https://github.com/davidz627)) 481 * Bugfix: csi plugin supporting raw block that does not need attach mounted failed ([#79920](https://github.com/kubernetes/kubernetes/pull/79920), [@cwdsuzhou](https://github.com/cwdsuzhou)) 482 * changes timeout value in csi plugin from 15s to 2min which fixes the timeout issue ([#79529](https://github.com/kubernetes/kubernetes/pull/79529), [@andyzhangx](https://github.com/andyzhangx)) 483 * The AWS credential provider can now obtain ECR credentials even without the AWS cloud provider or being on an EC2 instance. Additionally, AWS credential provider caching has been improved to honor the ECR credential timeout. ([#75587](https://github.com/kubernetes/kubernetes/pull/75587), [@tiffanyfay](https://github.com/tiffanyfay)) 484 * Reduces GCE PD Node Attach Limits by 1 since the node boot disk is considered an attachable disk ([#80923](https://github.com/kubernetes/kubernetes/pull/80923), [@davidz627](https://github.com/davidz627)) 485 486 487 488 # v1.14.5 489 490 [Documentation](https://docs.k8s.io) 491 492 ## Downloads for v1.14.5 493 494 495 filename | sha512 hash 496 -------- | ----------- 497 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes.tar.gz) | `8e529c27d8ce17bb35061870d19135fa7fde84790afc2c3780ecce524da8f3e61b4edcc0c60e83a1ea776c485a7acd48a037ff7e4e684423e7b19c682e9ba315` 498 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-src.tar.gz) | `0fa3a20b5aad9204507eba2cc6a5126170c652a7d59369dd1da8fa652a3e2052d1fdfd678a7e1762aef83cde5d4438092e3d30a5c2a2ec0ed4260d7b13324fa8` 499 500 ### Client Binaries 501 502 filename | sha512 hash 503 -------- | ----------- 504 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-darwin-386.tar.gz) | `a3b92e34c5e7e9679e306b5fbdea06611b5b0e775db28091ed64afd6f9b930bd7641f6ab69ba5dd128bbf3136cf3bf1d85d75ca16488281b7bcca409651caa8f` 505 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-darwin-amd64.tar.gz) | `79e7e1b5dc257d5358f994e518fa1c5dfae789a81ab936c6dec22e8bb87b364b1949d6b8fe1ec216c83cb0c5d3e22572c5198cf987ebecd14d2568fe89a09870` 506 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-linux-386.tar.gz) | `e0a822d551b0b6634179694f1c188ea716e3ceded9bf268f19c6fcc397013476f0d2b5ae199d9408c886d108d60693e9b9f03769ecd19a8f599f3d7802d39d2d` 507 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-linux-amd64.tar.gz) | `c56527f9378c16e90800b59d2bb2aa535d7ecb731edd8fb5e5824a5ff8e0d43b3ebfc1187628b38d6288c57369e7a0c77d70be17987ac03a8b7788357037edf3` 508 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-linux-arm.tar.gz) | `23d09797a0927b2f64ca1ffabac5131a2b013fa1ca85879681f75f83d242326e3db9b359816f905617727ef6a2ac7009055b050923467408ff47f0ba57fb640c` 509 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-linux-arm64.tar.gz) | `3fba6e0c1951f742862d8ead4a361445c8b663fcf605d2d3bdb17c5ca911f54b7944e3f64186793c6e514837b997bce335879032196ff52b39c50ea6cfcd0b79` 510 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-linux-ppc64le.tar.gz) | `0481eaa0d8c437511b9edb61338812b3dfac5b9a51a1a5cc1c88d9a735c66ea422fbf5a3a3a3479345f57484020e56dd735092b08196ef16ab22fe4341d5e09d` 511 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-linux-s390x.tar.gz) | `b4e291a966893949bae2d4752a7466d4bfc33265cf95f9caa677eb1e8f83341de378b9308a6b9ed5259a635b635d4fd004d4fdbf87ba6f92c2cae8b9317a3251` 512 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-windows-386.tar.gz) | `0f34e1d80e57730daa93d94a375b8b0fe5a571997ee8ce28185684628b1675ae1e8573f68bf168c50b4b46e3b5da76cacf73b44195a03a688e7763b46ffea9da` 513 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-client-windows-amd64.tar.gz) | `b00ee44e8af93986a3c27da355e5491709b777aa1e68d7f4fa0c632389602f4aa6b07527a075680cc83e903ad2b8493315e18fdad862a08fcdef76308e18c2f9` 514 515 ### Server Binaries 516 517 filename | sha512 hash 518 -------- | ----------- 519 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-server-linux-amd64.tar.gz) | `69d84f7885829db927205ddb49a4800c708de2922b04f65e85e4052988425c162c031c33553b669e772963d4bda9c12584d493da4ea00ca5123e33aecea3cc83` 520 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-server-linux-arm.tar.gz) | `37de1da7bfdbdbc7fd39e6a1f992f44504140bbdd0b36b0a62f16c38d3a172b1da1b86d6cff43a454f6e600708cd06daa34bca6c6f6e37d99ea2f14adeb64bcf` 521 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-server-linux-arm64.tar.gz) | `051b2cf5753792a58d25a183914c79be57a9a5322057910bafb223294b18030f4c6b61f2bee48d858ac7a3b41eb65370d991663dac4c24653fcb2523afa9063b` 522 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-server-linux-ppc64le.tar.gz) | `44805eb697eb97833d3c94482bfbc2e5b50acadff9db9b1cb63278990f6afcff5e50a9d9b6024e8d019494fbccf320755216db4f6d7e9a2605daad139c480393` 523 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-server-linux-s390x.tar.gz) | `024426ecc8f8a0e3ea64d506d678c092a92cc5f90ed998a9d55c7c2e5bf201144f4708bf46187786c58e672acadf20ae7bb8fb18570698f9f96ee05d9fd6ef25` 524 525 ### Node Binaries 526 527 filename | sha512 hash 528 -------- | ----------- 529 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-node-linux-amd64.tar.gz) | `84bc3a6aa82bc2a56d8f37e1abc1b572eaa7a0d85f242b9225c336b57164af3cccc31a4aa585f576875211c31bcf04522f774334e63cb7b95e577fb1c6612251` 530 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-node-linux-arm.tar.gz) | `a7364bb73a000e714478c66fbe1a7094e6e98eed58309fdf0af28d049541f1b2ccba65ebe4774d3e304356abf66c41d66040271206539f5b59dc7b9fd9fb4cef` 531 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-node-linux-arm64.tar.gz) | `2b30640bd37abbf14fb9c887ef62cf280bd397143b55816f76b44ba5912bece2516c1a4dbd40a4c12fe236e2388b5369b64e5ab779575dc5605d13259945266f` 532 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-node-linux-ppc64le.tar.gz) | `1cd0f330f33e476b20de5baffe7cb3b2935670b7ac3b14055af9ef751f697e39bd963be882b746d959a4352bc908c736b2d067e6b5511f5e1033f603f5cd6ebf` 533 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-node-linux-s390x.tar.gz) | `67a53cf0ab91cf1c7dc9c8b0bb941a463325e9673380890e9d1a6a61a6ff1e2e4ade53944ff19fb1deffe61ec4366943bebcbc651977bde20b43e5f931444e61` 534 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.5/kubernetes-node-windows-amd64.tar.gz) | `9e34d7158273cbd5d329ab4f13ea92f9c4d793e0b72c9a10d9d466e2894f86befbeeacf6b6e9506b950eaec773a2a38c1385de93f2d3cc53467a37e05815ad15` 535 536 ## Changelog since v1.14.4 537 538 * Fix CVE-2019-11249: Incomplete fixes for CVE-2019-1002101 and CVE-2019-11246, kubectl cp potential directory traversal ([#80436](https://github.com/kubernetes/kubernetes/pull/80436)) 539 * Fix CVE-2019-11247: API server allows access to custom resources via wrong scope ([#80750](https://github.com/kubernetes/kubernetes/pull/80750)) 540 541 See also the [security announcement for this release](https://groups.google.com/forum/#!topic/kubernetes-security-announce/vUtEcSEY6SM). 542 543 # v1.14.4 544 545 [Documentation](https://docs.k8s.io) 546 547 ## Downloads for v1.14.4 548 549 550 filename | sha512 hash 551 -------- | ----------- 552 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes.tar.gz) | `6a25452a4b248c1a0ffbc0ea19316791e10255e5d8f2a54a211a8463b2219048bded45bad0c7f41f91817a03a92e88fba0e5983554ededd0949eb77675493586` 553 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-src.tar.gz) | `5ab052b819c3d3fdb6149d597d3f638d9399b33868ebb76c4430ba34d741e9be0a23f295bd0a848745d2c2e565100e1dd53470708e98381934c14d9b40378c0b` 554 555 ### Client Binaries 556 557 filename | sha512 hash 558 -------- | ----------- 559 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-darwin-386.tar.gz) | `73e611d1da3f4acf263a1e9c0b2bdd5a052eac2a28551e01db202cd7abaa1951ea6af34319980c912c6899ccd98c0a5836041ee340f75645b8a1753fc8f4f4c2` 560 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-darwin-amd64.tar.gz) | `01fefb89be8d5615f365e0a9860dd72e69b12b14d83272f599a9806fc652805baca075c5e08780aa9afa710e852ee07d8eb270a9c5b78eaca6f81ec3539d7e8f` 561 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-linux-386.tar.gz) | `9498c346f8f2202f220b2ad86bff8637d7de49183279fc72a21af44a6475fcb991062d610cc4b03ee7bad2d18ebe25b05c519b06e67443b334cf061ac71890f7` 562 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-linux-amd64.tar.gz) | `5e3bbc8de68a0c25b84b1160ce874a275447a206ce4ed851d076c108c231405437c99eab0034ffeebc33a4f62e3253f79f652f03ee89e18fb8275cc1852be56a` 563 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-linux-arm.tar.gz) | `f15fa86947ef71490b5237f1f629f99cddc115626fcd88106efaa6d6a16be607df72049238f60e9ba77ae4c1a3d3dbd05793c206b5dea3a55371176527bee9ba` 564 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-linux-arm64.tar.gz) | `cdbd6efac49e58ae582d9e3cfa2ac90f2d6dca6271136847f88d582a627d1bb7233d816bd6ca025daf218d7030fe81f784524f142ab99b765861bfd904eb55ce` 565 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-linux-ppc64le.tar.gz) | `43da65bf491dcb12b7f7ab9a3f3db2c933a40e1601302ab8482efe3458ad6570429bdcb700dbd196b6f911cebc08eda83d61be3f86240241c5139ff3acd22def` 566 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-linux-s390x.tar.gz) | `6de5d190edc57dcf602d91da8c44ab5481d614791e64ca697b9393b53d52b861ff171a1c565af5774a4d28d7ea94c69c846f456aa7b77373e4b7ec3d3690f5af` 567 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-windows-386.tar.gz) | `f7c8b584de9662831f3082bbc8bdfbdf0a2df5056aded12febca843045cb5ff570b303ab002c2e8ae46752415d4acb2926ddfdd6a0c50ab624216eaf40e0610c` 568 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-client-windows-amd64.tar.gz) | `0136940266a354642d50fc4aa24ee1e14cf5098b142f8837041c19bba4626f2979e1efa9e559a22e32491f9d3279ee553242dad359c5b637f1b3e11891f97916` 569 570 ### Server Binaries 571 572 filename | sha512 hash 573 -------- | ----------- 574 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-server-linux-amd64.tar.gz) | `0405b9f0e1472f47739ec8e3d01694c992a3f0debe0f2fb233855a23bc55e66f5da0cc5a5f1927f807eae32aca39f57602cdfd78a51daa9a3b70ff146c0b2af6` 575 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-server-linux-arm.tar.gz) | `363105ee4895aa2b9df5ddb42fc4c726a16765dc02c1eb6c4dd3207e3f7726d7cf7da33fc48c7fb8681c3e6280bd893af5394f4d028a538c4c2c4991fe22aa61` 576 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-server-linux-arm64.tar.gz) | `bc8417511f908786240ba295d061ddcaad2fdb60708e6f3bbbf93087cfc1ee2299a377686e2e9957d2c626e919df81dd16e7596ce84790bc425e3f95201821e0` 577 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-server-linux-ppc64le.tar.gz) | `969b128a135c001d48a1652df9b1973769b4b692774247fb8ce2c7aa839d6ed815a03a34fe225b273bc82d8c32997944bbc84fc89dfbd053f7320c1d00f1163d` 578 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-server-linux-s390x.tar.gz) | `1492f504541ff3490b1b6e8c2450ae37c9931db5ad8609e1052c7baf99c8f8ff33739cdc00a4df12a89b38956cf79348c04f35a2806b5197204ca08b1da5236f` 579 580 ### Node Binaries 581 582 filename | sha512 hash 583 -------- | ----------- 584 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-node-linux-amd64.tar.gz) | `7397c1005a9ccadfcd46bd158f3a246882c2d47b5a431bf378d5dcc9c99a16c757787d175692ee3c8cc2a56cd401761d4a92651163786bfc41b7d438e74847fc` 585 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-node-linux-arm.tar.gz) | `5a4b7d20b5485249c4a68060eea1a17233d706412250a9d20f0f62d0c5b9ea6d3a89370eaf3f562062cb6212d493d50c5e44bda87e6ee7b0fc89c969a085f4a0` 586 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-node-linux-arm64.tar.gz) | `c413ec8b454f3f833f71de6512232ae1c6606fe556b6fe84ec3ffdb5e928691872b013f48c6b281f084901a434da178cab3d146aefb6a6b1ce459bcff0c21886` 587 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-node-linux-ppc64le.tar.gz) | `f60cf119c1a704201d44034922fd15bff832eec8b078868219cd38f91cc4850bf77a3827605f0b6f79e9406e0a0ec411f55ff0d99b034a912ddad62fbe3ac12f` 588 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-node-linux-s390x.tar.gz) | `0bafef46dcb4c8dbf88062a228ae6730b29d7536655b07c3fb5283c56508f2c820c6beacc962d1c2de20695fb6db655a154f6dd74c7a0784042f20412902ee6e` 589 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.4/kubernetes-node-windows-amd64.tar.gz) | `55d93e0f25d33970cfe05b63b9fd4b84e310230456ba1a32de6d3fd3ea87aa2f402672be0fea1910d6284f758af1c91ab00bd2cdfc30ca494e604de8d3256ae8` 590 591 ## Changelog since v1.14.3 592 593 ### Other notable changes 594 595 * fix kubelet fail to delete orphaned pod directory when the kubelet's pods directory (default is "/var/lib/kubelet/pods") symbolically links to another disk device's directory ([#79094](https://github.com/kubernetes/kubernetes/pull/79094), [@gaorong](https://github.com/gaorong)) 596 * Fix possible fd leak and closing of dirs in doSafeMakeDir ([#79534](https://github.com/kubernetes/kubernetes/pull/79534), [@odinuge](https://github.com/odinuge)) 597 * Default resourceGroup should be used when the value of annotation azure-load-balancer-resource-group is an empty string. ([#79514](https://github.com/kubernetes/kubernetes/pull/79514), [@feiskyer](https://github.com/feiskyer)) 598 * Remove pids cgroup controller requirement when related feature gates are disabled ([#79073](https://github.com/kubernetes/kubernetes/pull/79073), [@rafatio](https://github.com/rafatio)) 599 * Fix a bug where kubelet would not retry pod sandbox creation when the restart policy of the pod is Never ([#79451](https://github.com/kubernetes/kubernetes/pull/79451), [@yujuhong](https://github.com/yujuhong)) 600 * Bump ip-masq-agent to v2.3.0 to fix vulnerabilities. ([#77832](https://github.com/kubernetes/kubernetes/pull/77832), [@anfernee](https://github.com/anfernee)) 601 * fix pod stuck issue due to corrupt mnt point in flexvol plugin, call Unmount if PathExists returns any error ([#75234](https://github.com/kubernetes/kubernetes/pull/75234), [@andyzhangx](https://github.com/andyzhangx)) 602 * vSphere: allow SAML token delegation (required for Zones support) ([#78876](https://github.com/kubernetes/kubernetes/pull/78876), [@dougm](https://github.com/dougm)) 603 * fix: retry detach azure disk issue ([#78700](https://github.com/kubernetes/kubernetes/pull/78700), [@andyzhangx](https://github.com/andyzhangx)) 604 * try to only update vm if detach a non-existing disk when got <200, error> after detach disk operation 605 * This adds some useful metrics around pending changes and last successful ([#78602](https://github.com/kubernetes/kubernetes/pull/78602), [@paulgmiller](https://github.com/paulgmiller)) 606 * sync time. 607 608 * The goal is for administrators to be able to alert on proxies that, for 609 * whatever reason, are quite stale. 610 * Fix a string comparison bug in IPVS graceful termination where UDP real servers are not deleted. ([#78999](https://github.com/kubernetes/kubernetes/pull/78999), [@andrewsykim](https://github.com/andrewsykim)) 611 * Resolves spurious rollouts of workload controllers when upgrading the API server, due to incorrect defaulting of an alpha procMount field in pods ([#78883](https://github.com/kubernetes/kubernetes/pull/78883), [@liggitt](https://github.com/liggitt)) 612 * Fixes a memory leak in Kubelet on Windows caused by not not closing containers when fetching container metrics ([#78594](https://github.com/kubernetes/kubernetes/pull/78594), [@benmoss](https://github.com/benmoss)) 613 614 615 616 # v1.14.3 617 618 [Documentation](https://docs.k8s.io) 619 620 ## Downloads for v1.14.3 621 622 623 filename | sha512 hash 624 -------- | ----------- 625 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes.tar.gz) | `7e82dc2070b55ca8f4746579a66d9dcd61e750f8c20632f901cbeb8c9c5bc28b7c428531d96375a5491e22397e4da3abaad2b1399250736380c586a3163fc960` 626 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-src.tar.gz) | `58db2d138b1a0575dfdc9be1a742788c56626275f588d262ab64840627e4055c68480f3b2b96c0059967e13f53028cc0bae5882f4abf1767bfcea15ebb761c73` 627 628 ### Client Binaries 629 630 filename | sha512 hash 631 -------- | ----------- 632 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-darwin-386.tar.gz) | `29ee18e669171002a11b39fe5046577d91472dfb089de95ad23499a8eb686777887c0bcc81a1681b7ad28f35303ed638bfef36ec94fc19caec9982ba94cd9c74` 633 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-darwin-amd64.tar.gz) | `2cf1680e9dfae84dbf4d1f10e9142a087ac9be56c9c5233b0cea669f9dae59ca645e23109a5f039c174dea42e035180e9e3b5298affcba4aa980760af16ba2ff` 634 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-linux-386.tar.gz) | `39ace58bdb85397b856b52d54e97eb9f489169995f55a55b278c1c923829c0c264113a3251f01b705e53f35a213d420570feff41c130935aca3ce9e0c1bccc5b` 635 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-linux-amd64.tar.gz) | `dcb6eeebf618a01f274091998984f8afca567f7a20ffff810bef97e9654af5e4b912f83c97e6ed13dd1b93fcea2c14d7cde4fd55b174f32aa1f2f005baadf2d7` 636 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-linux-arm.tar.gz) | `4a7dd3310fa96e842ab3505f7815b0778af621fd639ab8296b4994428d930d63057f073e81ebc29852beaa4f6e55271289342c90d46d1b5a70564c58f1bd61c7` 637 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-linux-arm64.tar.gz) | `7f091cff9c63d58623b5b49967b8824e02139a347c9c5eb9c9fa178b3825320614623afc2e0a2414d29b3cb8f70c8ecfbb23c907e351c63312cd4bc4a57d4610` 638 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-linux-ppc64le.tar.gz) | `f7f43d71663a19c2526ae8496b76b2e9df2285ae31f94ba054e45704cc94e2e04cc563fdfce6afebd532ecaf2fb2e68e7ed1bd2c39641ac10bdf910cb17df3ca` 639 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-linux-s390x.tar.gz) | `23e782ab1fe8e5d30e1cdff8594de4cc4750ab178ae3c71c177c49acf9f35d908bdf8281ecb174a52664b9841ece206ec8140901334ef7836f556fbe66c04572` 640 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-windows-386.tar.gz) | `25c05c105996c0d75443e75b0809b7bf7c33d6d8d1d24d2b0b9c2f872838277030870c02c96bcd4ac378c66a7ad59e2165293a6ac6feea12691d68749dfe1c5b` 641 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-client-windows-amd64.tar.gz) | `df0da8deca3a354406a4be3f025bd9b6729f68c4414126bdcf58a649228196de42d8b94227963bece055801cf883a5de3cbb68b940bfdc98c56eab26e3015029` 642 643 ### Server Binaries 644 645 filename | sha512 hash 646 -------- | ----------- 647 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-server-linux-amd64.tar.gz) | `8d7192d1e74a139feca532d9956f6d3dc5289eb5c3eff643fd13d02815ed11a12dca5580e830d69cb142e1b5e99e0ecf8939d9e260888bca184f9e013f2b5948` 648 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-server-linux-arm.tar.gz) | `89c26c83562e32340ab58a0aa835c0b586c6626e811231129645205edfc0bf563aba4125bb932761a27757a0f205ea013fea30210448eac68cd5b25787b60bc8` 649 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-server-linux-arm64.tar.gz) | `f74ef5b4a680167251b93d346c9864436f3906f72d676483c9ce27ea0f42e366eab0f85aebd138b915fae2e893bed579a755be9869bb0fce43c0460cd629738c` 650 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-server-linux-ppc64le.tar.gz) | `6bfc3cbd4604112de5a09fa9c8e35ad4637d1fedc024ef20fafb9ef5adf5f160656a6e7c3298d7f39f741872dc496e5f1b3dc422b5e6f24be7b1d20704e80f6b` 651 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-server-linux-s390x.tar.gz) | `a1355a9db2e34b062b5e335c52e7a5e2b6e55490896bf9ec1f3900b048b73d4c9eeac6fd770efed269a7fbdcdc613ba10fce3ce4d22aa6d86609790b91c321e9` 652 653 ### Node Binaries 654 655 filename | sha512 hash 656 -------- | ----------- 657 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-node-linux-amd64.tar.gz) | `1eb8610f93153b5e274ae61f68f9ccd30bdb5373b6098b03a0f885da4b389bfd2f060208aedab663c03311bcae853feed50070e55497d353b243e781d7cef62a` 658 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-node-linux-arm.tar.gz) | `cedef35f9e84c93d099117a413de39bb5a55e532de1b24dd6c53bcf849ed3100321a0edbd1b895d0a82c0cbdf8d567a44818c9b5d7fd064a168f42a300c834e0` 659 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-node-linux-arm64.tar.gz) | `d1f4152ece68cd3d96ff6632fd2229f7da630d5256fd8edd225ee75a4fc60976ad0778dfc71dee8bafdf6b1373a365a157f17faec00fdf8ce87664ace99998f2` 660 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-node-linux-ppc64le.tar.gz) | `dbfc833c385c46118aac8d651b9053e48ee9f961fee0668756db0a1254d178d7c9d6c160e6dc9c14e643a1831a9d0e989a00a1b78b455b822c3db8ba29f6b223` 661 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-node-linux-s390x.tar.gz) | `d715253ab6d2b6b6478d19f4ee4b69d4ba4ae5f20643c3b908fe6e481900f83440ded286c955aa1b189bf4ea6461a0d9e4348d61205dc45e1ef26f8b2c03326b` 662 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.3/kubernetes-node-windows-amd64.tar.gz) | `3e111f2121c4d87d9492c8ba13812537cce11bf2c52852715d0ee5a3d475981567a058b0a17de9f65e1ba09cb5e1b4b1a78c530d78850a0edfe610b77a7dca18` 663 664 ## Changelog since v1.14.2 665 666 ### Other notable changes 667 668 * Fix vSphere SAML token auth when using Zones ([#78180](https://github.com/kubernetes/kubernetes/pull/78180), [@dougm](https://github.com/dougm)) 669 * IPVS: Disable graceful termination for UDP traffic to solve issues with high number of UDP connections (DNS / syslog in particular) ([#77802](https://github.com/kubernetes/kubernetes/pull/77802), [@lbernail](https://github.com/lbernail)) 670 * Fix broken detection of non-root image user ID ([#78261](https://github.com/kubernetes/kubernetes/pull/78261), [@tallclair](https://github.com/tallclair)) 671 * fix azure retry issue when return 2XX with error ([#78298](https://github.com/kubernetes/kubernetes/pull/78298), [@andyzhangx](https://github.com/andyzhangx)) 672 * kubelet: fix fail to close kubelet->API connections on heartbeat failure when bootstrapping or client certificate rotation is disabled ([#78016](https://github.com/kubernetes/kubernetes/pull/78016), [@gaorong](https://github.com/gaorong)) 673 * Active watches of custom resources now terminate properly if the CRD is modified. ([#78029](https://github.com/kubernetes/kubernetes/pull/78029), [@liggitt](https://github.com/liggitt)) 674 * client-go and kubectl no longer write cached discovery files with world-accessible file permissions ([#77874](https://github.com/kubernetes/kubernetes/pull/77874), [@yuchengwu](https://github.com/yuchengwu)) 675 * Fix panic logspam when running kubelet in standalone mode. ([#77888](https://github.com/kubernetes/kubernetes/pull/77888), [@tallclair](https://github.com/tallclair)) 676 * Fixed a bug in the apiserver storage that could cause just-added finalizers to be ignored on an immediately following delete request, leading to premature deletion. ([#77619](https://github.com/kubernetes/kubernetes/pull/77619), [@caesarxuchao](https://github.com/caesarxuchao)) 677 * Fixes a bug where dry-run is not honored for pod/eviction sub-resource. ([#76969](https://github.com/kubernetes/kubernetes/pull/76969), [@apelisse](https://github.com/apelisse)) 678 * fix incorrect prometheus azure metrics ([#77722](https://github.com/kubernetes/kubernetes/pull/77722), [@andyzhangx](https://github.com/andyzhangx)) 679 * fix KUBE_SERVER_PLATFORMS null error when cross compiling kubectl for non-linux platform ([#78059](https://github.com/kubernetes/kubernetes/pull/78059), [@figo](https://github.com/figo)) 680 681 682 683 # v1.14.2 684 685 [Documentation](https://docs.k8s.io) 686 687 ## Downloads for v1.14.2 688 689 690 filename | sha512 hash 691 -------- | ----------- 692 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes.tar.gz) | `ef1228ef7cdc3a53e9a5003acb1616aff48eba53db147af82c5e318c174f14db410bb55c030acd67d7f7694b085185ca5f9ac1d3fb9bb6ec853196571e86ad2e` 693 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-src.tar.gz) | `1721ea726dd19f06bade3e9751379764ffb16289b8902164d78a000eb22da15f11358b208f3996df09cd805f98daa540e49f156c1b7aabee6a06df13de8386ca` 694 695 ### Client Binaries 696 697 filename | sha512 hash 698 -------- | ----------- 699 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-darwin-386.tar.gz) | `f707f3293173cbb47dc8537b19d7da443e40d9c2b3945e8e0559513d227d98a97058b5ee3762fbf93e79b98bceadb23fc985bfbff33c8f4970966383d5032df1` 700 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-darwin-amd64.tar.gz) | `dcd61588fc0b27d6539f937106a88f8ebb3f19e9a41d37a79804a2594e12860247883374d7594b52a248915820be98b0dd7f756e581f5512cf731f9992bc3950` 701 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-linux-386.tar.gz) | `90ad262988898cc25c2d84fdf1d62d3cdf8f16a9b7598d477a1b516b7e87e19196a4e501388e68fccc30916ac617977f6e22e4ec13fa2046bda47d386b45a0e6` 702 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-linux-amd64.tar.gz) | `a4394293cecdc177db7d3ef29f9d9efb7f922d193b00d83fa17c847e2aa1cd1c38eff1f4233843fededf15d99f7c434bf701d84b93a3cb834a4699cbddf02385` 703 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-linux-arm.tar.gz) | `265599b200f6de8d2c01ac36a33a0fca9faf36fb68e3e3dd5dad9166b9e6605db2aadd4199a05b5b9e20d065a8e59e7d0d130e5038dc01b37ed9705a8550d677` 704 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-linux-arm64.tar.gz) | `31799018b7840cafac0fa4f8cc474396feaab71340eb7f38a122109fdcf759afc6066e67c5a26fe234232ab9a180d7312e81b3911c153f2e949415236a7b1709` 705 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-linux-ppc64le.tar.gz) | `670bbe7c3142ccfa99a1eebc6d94798a8c3720165301ef615812aea64e512e03db4a9e2d80bfa073083b87c1a123a1a8e0c72fe2be26e2dfe8a499a3237deb32` 706 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-linux-s390x.tar.gz) | `58d161e747ec0924f3a937bd4e6856be9bad9227ca2564f2b59cdc9bfd063d78cb9c6381441aac21d3d809a1edee059697cbef5aabd344bb3fb58d4a56641415` 707 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-windows-386.tar.gz) | `696caeb9611137bce1988c8bf7a1e326f92dbb6f9eb31f82cc2d9cf262888b220c3abed5edb8807c58d37b659a80e46f79ecb9d8ea67627cf6a7e6b9ffa3e5c6` 708 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-client-windows-amd64.tar.gz) | `156ccc2102a6f92fe1078feaed835913b34eac94bbd0846726eb43fa60f0beb724355e3a3be4de87630f27f67effdd88a5014aa197ba8695bf36da2b70ee1c14` 709 710 ### Server Binaries 711 712 filename | sha512 hash 713 -------- | ----------- 714 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-server-linux-amd64.tar.gz) | `f7d9687eb49ea71f0d8b1ccfac33ed05cd341d7cfacb0711fce4a722801769deb05f72f19ade10b6dc29409f0c9136653c489653ca1f20b698c1310f8a43600f` 715 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-server-linux-arm.tar.gz) | `5c2247e4cab886cbca59ef47ea32d9ab8bb5f47495f844337dadce2362b76ebedc8a912f34131f9ec2e15bcb9023d75efb561ce7e51ce5fc7d0cb6f058a96840` 716 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-server-linux-arm64.tar.gz) | `a341bb15e659d4000fe29b88371cc1c02df4715786901b870546c04cd943f5cad56bd4f014062c4ef2d601f107038bb4024c029f62b8b37456bbcf4d14cfc5d0` 717 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-server-linux-ppc64le.tar.gz) | `d179c809da68cc4530910dd1a7c3749598bd40f5b7a773b2b3a9b9d0b0e25c5a0fa8f2caa8f1874b7168d2acb708f0d5014ca4f4721252ce414e36734485e32b` 718 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-server-linux-s390x.tar.gz) | `fdc8ffccd1c5a2e225f19b52eabceae5e8fac5e599235797b96d37223df10d45f70218dcbf5027a00db0129929fe179cd16b1f42ae2a6e7a4d020a642cd03981` 719 720 ### Node Binaries 721 722 filename | sha512 hash 723 -------- | ----------- 724 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-node-linux-amd64.tar.gz) | `12c6139a4b497220f239f6c5e9a9b2e864d6dc298495ef4243b780fcf6c9c3aab53c88fa33d8527ed45d79de707cbce733e0c34c06b10fe2a07b4c3daafc0f50` 725 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-node-linux-arm.tar.gz) | `53e14c9dd53562747dcfdfff7738bccdd369a2bd6f550e1ce181aa219e48c0fe92f786c4ed8d4f62fada48018917d573e4e63c0168bf205b707309ef78bac9b5` 726 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-node-linux-arm64.tar.gz) | `5917436bdafab57f6564d6e32819b28f32d373bdb22ae53a46f7c7510283ffa87199d08db31862f8db286d5e96a37e299f8a31f0fd630bfd94698ba58b16e9af` 727 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-node-linux-ppc64le.tar.gz) | `12a8ca3c87f165ef4eb493adcd3038d5689c592b411ebbbc97741b1de67a40f91fed7c83d0bf97bd59719c8d08e686c49e6d6dd9c6ef24b80010eb0777614187` 728 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-node-linux-s390x.tar.gz) | `1919f8b370199261803ec856e558ad75100cf6db8f5619be5710f528a46a6c58692d659bb11525e351fd46673765348050ea6f1a7427fd458386f807040b67eb` 729 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.2/kubernetes-node-windows-amd64.tar.gz) | `86057b6ca519a6b454a4b898c7a12f12a2bb25c8be85e53fd2c9b1e4191e334611ca87e679b5832acdd37c05486972eb9c0b4c4bcbf4b688239d9482a9590745` 730 731 ## Changelog since v1.14.1 732 733 ### Other notable changes 734 735 * Update to use go 1.12.4 ([#76576](https://github.com/kubernetes/kubernetes/pull/76576), [@cblecker](https://github.com/cblecker)) 736 * Update to use go 1.12.5 ([#77528](https://github.com/kubernetes/kubernetes/pull/77528), [@cblecker](https://github.com/cblecker)) 737 * Check if container memory stats are available before accessing it ([#77656](https://github.com/kubernetes/kubernetes/pull/77656), [@yastij](https://github.com/yastij)) 738 * Bump addon-manager to v9.0.1 ([#77282](https://github.com/kubernetes/kubernetes/pull/77282), [@MrHohn](https://github.com/MrHohn)) 739 * Rebase image on debian-base:v1.0.0 740 * If a pod has a running instance, the stats of its previously terminated instances will not show up in the kubelet summary stats any more for CRI runtimes like containerd and cri-o. ([#77426](https://github.com/kubernetes/kubernetes/pull/77426), [@Random-Liu](https://github.com/Random-Liu)) 741 * This keeps the behavior consistent with Docker integration, and fixes an issue that some container Prometheus metrics don't work when there are summary stats for multiple instances of the same pod. 742 * Add name validation for dynamic client methods in client-go ([#75072](https://github.com/kubernetes/kubernetes/pull/75072), [@lblackstone](https://github.com/lblackstone)) 743 * Fix issue in Portworx volume driver causing controller manager to crash ([#76341](https://github.com/kubernetes/kubernetes/pull/76341), [@harsh-px](https://github.com/harsh-px)) 744 * Fixes segmentation fault issue with Protobuf library when log entries are deeply nested. ([#77224](https://github.com/kubernetes/kubernetes/pull/77224), [@qingling128](https://github.com/qingling128)) 745 * Update Cluster Autoscaler to 1.14.2 ([#77064](https://github.com/kubernetes/kubernetes/pull/77064), [@losipiuk](https://github.com/losipiuk)) 746 * https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.14.2 747 * https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.14.1 748 * Fixes an error with stuck informers when an etcd watch receives update or delete events with missing data ([#76675](https://github.com/kubernetes/kubernetes/pull/76675), [@ryanmcnamara](https://github.com/ryanmcnamara)) 749 * [fluentd-gcp addon] Bump fluentd-gcp-scaler to v0.5.2 to pick up security fixes. ([#76762](https://github.com/kubernetes/kubernetes/pull/76762), [@serathius](https://github.com/serathius)) 750 * specify azure file share name in azure file plugin ([#76988](https://github.com/kubernetes/kubernetes/pull/76988), [@andyzhangx](https://github.com/andyzhangx)) 751 * Windows nodes on GCE use a known-working 1809 image rather than the latest 1809 image. ([#76722](https://github.com/kubernetes/kubernetes/pull/76722), [@pjh](https://github.com/pjh)) 752 * kube-proxy: os exit when CleanupAndExit is set to true ([#76732](https://github.com/kubernetes/kubernetes/pull/76732), [@JieJhih](https://github.com/JieJhih)) 753 * Clean links handling in cp's tar code ([#76788](https://github.com/kubernetes/kubernetes/pull/76788), [@soltysh](https://github.com/soltysh)) 754 * Adds a new "storage_operation_status_count" metric for kube-controller-manager and kubelet to count success and error statues. ([#75750](https://github.com/kubernetes/kubernetes/pull/75750), [@msau42](https://github.com/msau42)) 755 * kubeadm: Fix a bug where if couple of CRIs are installed a user override of the CRI during join (via kubeadm join --cri-socket ...) is ignored and kubeadm bails out with an error ([#76505](https://github.com/kubernetes/kubernetes/pull/76505), [@rosti](https://github.com/rosti)) 756 * fix detach azure disk back off issue which has too big lock in failure retry condition ([#76573](https://github.com/kubernetes/kubernetes/pull/76573), [@andyzhangx](https://github.com/andyzhangx)) 757 * Ensure the backend pools are set correctly for Azure SLB with multiple backend pools (e.g. outbound rules) ([#76691](https://github.com/kubernetes/kubernetes/pull/76691), [@feiskyer](https://github.com/feiskyer)) 758 * fix azure disk list corruption issue ([#77187](https://github.com/kubernetes/kubernetes/pull/77187), [@andyzhangx](https://github.com/andyzhangx)) 759 * [IPVS] Introduces flag ipvs-strict-arp to configure stricter ARP sysctls, defaulting to false to preserve existing behaviors. This was enabled by default in 1.13.0, which impacted a few CNI plugins. ([#75295](https://github.com/kubernetes/kubernetes/pull/75295), [@lbernail](https://github.com/lbernail)) 760 * [metrics-server addon] Restore connecting to nodes via IP addresses ([#76819](https://github.com/kubernetes/kubernetes/pull/76819), [@serathius](https://github.com/serathius)) 761 * Fixes a NPD bug on GCI, so that it disables glog writing to files for log-counter ([#76211](https://github.com/kubernetes/kubernetes/pull/76211), [@wangzhen127](https://github.com/wangzhen127)) 762 * Fixes bug in DaemonSetController causing it to stop processing some DaemonSets for 5 minutes after node removal. ([#76060](https://github.com/kubernetes/kubernetes/pull/76060), [@krzysztof-jastrzebski](https://github.com/krzysztof-jastrzebski)) 763 764 765 766 # v1.14.1 767 768 [Documentation](https://docs.k8s.io) 769 770 ## Downloads for v1.14.1 771 772 773 filename | sha512 hash 774 -------- | ----------- 775 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes.tar.gz) | `87f4cb0c6c137cbd07cc0e0b7049722ed2a3f21866ac02aecf1a0e03d350d1e90d7907487bac5ef224da75d05056abfa39f5b1b3741987dde1c5165f379cb253` 776 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-src.tar.gz) | `ef1b4ed78ed92124fbec2b2bf54ba3b293038f9c8f1df3883071ae9430450cab9c02c6111cf171ad8d61a0aef6d236fbb9f0f1526e6c00f0452323e8c7c16305` 777 778 ### Client Binaries 779 780 filename | sha512 hash 781 -------- | ----------- 782 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-darwin-386.tar.gz) | `f9f14293ab8c6e6d49c5be16c2bcfad640a258d3e1ce600d6b327a4ba84c369f679b8ed65f7f96c23b7277c6cbf4fa54cc75dd8d75e4c8a3b756dc02f7e99071` 783 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-darwin-amd64.tar.gz) | `761c6334ff33e0487feb15f9c335e0e44f36fbb1b5b95ddb8aad0383a085ce5c31f762d60b2fc4365052221b7594b5e9c046c25c9806ca93e7af9183e4474cb2` 784 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-linux-386.tar.gz) | `58c4db0219debd85ded6dd0eac2ceac3432772805826b362d915571aec0b3f93e60eaee7181bbf28bf7fb7d93011b9849fa486f7a05b53f4ac922845f2a5deeb` 785 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-linux-amd64.tar.gz) | `66c3a470caecfb35ce9a995a6298e689aed5fabefbdb8aca5086adff572266ae47b997eea03ff3ce0272fdb5be8e22aced3e3ae35906b5ac90cf928d7c0c974f` 786 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-linux-arm.tar.gz) | `50f76e9cca5e056d9dabe7f27de7db72539cb33c3e24bb541e35cf32293b7614d4a22447ec6d9e6a604bfe97825f023e72934993bf144c7763f76896d57595f6` 787 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-linux-arm64.tar.gz) | `088621d5cbb8587896b38e6e1faa84471490a1bd2371c082143aeebc0bac6498422c9175014cba22e5190dd761d4154bec91b1d1b93a09d1fae771d3bebf2227` 788 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-linux-ppc64le.tar.gz) | `d743819920dd3ac906a855af2c1a1327f991e4c295357c610b1fad5d5cd8abf5ac1296e3bf9a46fa3f8877a152e3f8fba3a5d27e51289926f7519215769c24c6` 789 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-linux-s390x.tar.gz) | `71cdd44a0d5418500407e9eea6f7118b7384b8c9a4bafaefb78c107b23e0503393b5a831bbe8eaaab6a37b4b23be3e7c5f700b991bbb4e656a72c46198e40e35` 790 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-windows-386.tar.gz) | `ffeacdc7658da5564811047e66e2b8e370d037b8d983a2af0ceb9cf762f6214658f633fe626d6e97810f520c664c0ab8d346a8e2ce6be330787c099693d76c83` 791 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-client-windows-amd64.tar.gz) | `f49b8acef5c31b59dfff0d63b4e175f54f605dd5783bdd57e18cdea766719860490281d2cdf0a3ea1f77d2c3753b4ec735af06ccda7f5ca4fcab14cd26411ef2` 792 793 ### Server Binaries 794 795 filename | sha512 hash 796 -------- | ----------- 797 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-server-linux-amd64.tar.gz) | `4775257f919bf013a92d6e3b66099d85e09be32b462189ede82874ea860ccacc3f42ff2d70e9e55b9f3b7886172bf85b26a87bc51e9d42435bfd608308b84ec6` 798 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-server-linux-arm.tar.gz) | `2806099d6bdd32940a3221a076fff51eb9c79729e058a6b5ef283cfbbf71695d35b0d121643282a089d1ce3ca8599e3a6d095ad90be931bd893ac6ddae562664` 799 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-server-linux-arm64.tar.gz) | `1aa3c44189f9be25181b69e6ef336955f92ceb0a35c07579e386e4c0584d4bbb5b6f7cb044ccb7002ea111f8286439f02679391f66799c78e8b9d6e100bee5e5` 800 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-server-linux-ppc64le.tar.gz) | `6e91be7bf7b3cb1dc20a395cbf32c70ad867f1300d09800bb2c6854c93ff8d9cf8c180b703f3172564f0b10336ce04740015f67738fa5401992ad6e3505b1b69` 801 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-server-linux-s390x.tar.gz) | `ee915182eda63d67292464ed3a3e029c1749d016cd730e15dc8fd19fdcc8ee5ae7bc926681935b6e6837d555e2165c231d3554e9428ac09b051b31f5d22b07e1` 802 803 ### Node Binaries 804 805 filename | sha512 hash 806 -------- | ----------- 807 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-node-linux-amd64.tar.gz) | `df2b5010eea175fd45868c908cc769cc8fefb367899af448ef9c64db5e4a7b50db9bdba77b81943446d0abeb2d9d36d72a22a8d72042f88eecb9123c9b77c0b5` 808 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-node-linux-arm.tar.gz) | `338ca18540c087c81b07bd341f390b78e446deb270d7e320ef108f9f293518c26580c17968c1a87fe7af2546ff56a9392009a354202dea1d2083b79652250da3` 809 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-node-linux-arm64.tar.gz) | `dd2544dd9543cb9a556def0116fdccb8b14c0e7ae07debbf10f3b4ac0669a1f38be28327114781157cc9ae06e96140b1a0650eeb707bd883ae3509e0ee873da7` 810 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-node-linux-ppc64le.tar.gz) | `866fd6680f0d35c747c54369e6363b0241a233a505110a899236e1390ec7365b9ae7df4ddf7087514dc5102ce757a46b4fb218db0c081bb15c200ed526209a83` 811 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-node-linux-s390x.tar.gz) | `87e7b803f1ae05bf4652fd0b2f636ce61bd1100e40ce7c5c2530407346260435a8f649a41bfbfa5cacb7a810d007ac19323056ef175f67aee469528b0a7d7e30` 812 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.1/kubernetes-node-windows-amd64.tar.gz) | `0e46f70d7f1ec542b33119a9773a98dfb46d80f96a9f31f22ccc33c902e4bb102e2d2453a0fcebcfe319b331d1a78606269816f0f239b68902f7059240ca790e` 813 814 ## Changelog since v1.14.0 815 816 ### Other notable changes 817 818 * GCE/Windows: disable stackdriver logging agent to prevent node startup failures ([#76099](https://github.com/kubernetes/kubernetes/pull/76099), [@yujuhong](https://github.com/yujuhong)) 819 * Support vSphere SAML token auth when using Zones ([#75515](https://github.com/kubernetes/kubernetes/pull/75515), [@dougm](https://github.com/dougm)) 820 * Fix empty array expansion error in cluster/gce/util.sh ([#76111](https://github.com/kubernetes/kubernetes/pull/76111), [@kewu1992](https://github.com/kewu1992)) 821 * [stackdriver addon] Bump prometheus-to-sd to v0.5.0 to pick up security fixes. ([#75362](https://github.com/kubernetes/kubernetes/pull/75362), [@serathius](https://github.com/serathius)) 822 * [fluentd-gcp addon] Bump fluentd-gcp-scaler to v0.5.1 to pick up security fixes. 823 * [fluentd-gcp addon] Bump event-exporter to v0.2.4 to pick up security fixes. 824 * [fluentd-gcp addon] Bump prometheus-to-sd to v0.5.0 to pick up security fixes. 825 * [metatada-proxy addon] Bump prometheus-to-sd v0.5.0 to pick up security fixes. 826 * kube-proxy no longer automatically cleans up network rules created by running kube-proxy in other modes. If you are switching the mode that kube-proxy is in running in (EG: iptables to IPVS), you will need to run `kube-proxy --cleanup`, or restart the worker node (recommended) before restarting kube-proxy. ([#76109](https://github.com/kubernetes/kubernetes/pull/76109), [@vllry](https://github.com/vllry)) 827 * If you are not switching kube-proxy between different modes, this change should not require any action. 828 * This fixes a bug where restarting the iptables proxier can cause connections to fail (https://github.com/kubernetes/kubernetes/issues/75360). 829 * kubeadm: fixes error when upgrading from v1.13 to v1.14 clusters created with kubeadm v1.12. Please note that it is required to upgrade etcd during the final v1.13 to v1.14 upgrade. ([#75956](https://github.com/kubernetes/kubernetes/pull/75956), [@fabriziopandini](https://github.com/fabriziopandini)) 830 * Fixes a regression proxying responses from aggregated API servers which could cause watch requests to hang until the first event was received ([#75887](https://github.com/kubernetes/kubernetes/pull/75887), [@liggitt](https://github.com/liggitt)) 831 * Increased verbose level for local openapi aggregation logs to avoid flooding the log during normal operation ([#75781](https://github.com/kubernetes/kubernetes/pull/75781), [@roycaihw](https://github.com/roycaihw)) 832 * Update Cluster Autoscaler to 1.14.0; changelog: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.14.0 ([#75480](https://github.com/kubernetes/kubernetes/pull/75480), [@losipiuk](https://github.com/losipiuk)) 833 * Ensures the conformance test image saves results before exiting when ginkgo returns non-zero value. ([#76039](https://github.com/kubernetes/kubernetes/pull/76039), [@johnSchnake](https://github.com/johnSchnake)) 834 * GCE Windows nodes will rely solely on kubernetes and kube-proxy (and not the GCE agent) for network address management. ([#75855](https://github.com/kubernetes/kubernetes/pull/75855), [@pjh](https://github.com/pjh)) 835 * kubeadm: fix "upgrade plan" not defaulting to a "stable" version if no version argument is passed ([#75900](https://github.com/kubernetes/kubernetes/pull/75900), [@neolit123](https://github.com/neolit123)) 836 * kubeadm: preflight checks on external etcd certificates are now skipped when joining a control-plane node with automatic copy of cluster certificates (--certificate-key) ([#75847](https://github.com/kubernetes/kubernetes/pull/75847), [@fabriziopandini](https://github.com/fabriziopandini)) 837 * [IPVS] Allow for transparent kube-proxy restarts ([#75283](https://github.com/kubernetes/kubernetes/pull/75283), [@lbernail](https://github.com/lbernail)) 838 839 840 841 # v1.14.0 842 843 [Documentation](https://docs.k8s.io) 844 845 ## Downloads for v1.14.0 846 847 848 filename | sha512 hash 849 -------- | ----------- 850 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes.tar.gz) | `0ad264a46f185a9ff4db0393508a9598dab146f438b2cfdc7527592eb422870b8f26ade7ed089359c06741d998fcd730f897eae261f922c1a26d9fdc034d270d` 851 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-src.tar.gz) | `c5175439decc1c5f54254572bfec3c9f61f39d6bd1cbc28d1f771f8f931b98f0c305f1871618ce7e9de9cf3bf8227e19dcf985a7e017c74d0d7ab4005b3dbd59` 852 853 ### Client Binaries 854 855 filename | sha512 hash 856 -------- | ----------- 857 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-darwin-386.tar.gz) | `68bdba50a2b0be755e73e34ffc758fd419940adace096b1ddebd44a0eae2c7cdaed984965ea8f2145c1cab0be47bd6c72c2aeb73e51d449bfeb9ce1854b6c562` 858 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-darwin-amd64.tar.gz) | `255bd93082b3ac5d69bd4e45c75c9f19efee50ad6add50837ff2987ce16cbcc485fad334c980b17f69e5a344ee50548e206f747441ad4a045aa65746c79d10ca` 859 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-linux-386.tar.gz) | `2bd115ad2503fdfe5482e4592fcc0c8a2aee36be5205220a13c8050cd1e55dd3c08377425dbe5a03e4ffd21cf603c739ec4eaf3e5b2514a725d095df46f25d98` 860 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-linux-amd64.tar.gz) | `a551adf8019b17fce5aff2b379fab3627588978a2d628b64ba1af6f3be1b435322368b00dd04fa739d01c341420016b93239cc0d4601cee86706d81d78cb4d7f` 861 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-linux-arm.tar.gz) | `24e771cd4074786330e07f5537259a28d0932102639326230d9161f12a8dc545638a55bc252771eb4e21e95e2c7f0918dc1238ac4dc70d3b8b33f093da7123ab` 862 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-linux-arm64.tar.gz) | `16204f2345ab3523bbe3c868f04806a97c111d940b2594aaff67cf73b4259040c7770d5b0e7bdb7ffd7389f87e5f090ae875bd0f192b07582f59a01a1df32f5b` 863 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-linux-ppc64le.tar.gz) | `cd9ce829d585dd3331c53d35015d4017026d5efd24b9bc2f342995245628598c98bd8b1f1d706b196a7b3046a44049d4aba6efb4b1000722bfd055bd8a662f1f` 864 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-linux-s390x.tar.gz) | `482c0a8e53b27f8922f58d89fb81842ddd9c3ffd120e635838992dc97d535e46b42e7d8c439cb739b7c1d63c8eed27d7e3bcac7126a6a96e56cc13d52f396328` 865 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-windows-386.tar.gz) | `4446d666f999e979a7245e1b7ebf4817f7bd23aa247a38853a63b9cda473c7d4c2d376a2fd0df13ba15b740bf6b458cac14bd03dbf5a8151fc230e40c08294cf` 866 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-client-windows-amd64.tar.gz) | `97f4789f21d10fd3df446e55bc489472dcd534c623bb40dc3cb20fe1edd74c1a89a50ce7caa4e5e0536f3b22d8698060bfe8c46f4adbd0e507349412e52664e8` 867 868 ### Server Binaries 869 870 filename | sha512 hash 871 -------- | ----------- 872 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-server-linux-amd64.tar.gz) | `25739802a641517a8bbb933b69000a943e8dd38e616b8778149dd0138737abacf377683da2ff35fdd0bbb305b88bc8fc711df20a2585720a43bb674ef36b034f` 873 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-server-linux-arm.tar.gz) | `c1dbba77a4ff5661eb36c55182a753b88ccc9b89ca31e162b06672126743cfea115b2f8ea8658b12344c36df17958e310c1b8efbdd7800f44f013e1e6f10477d` 874 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-server-linux-arm64.tar.gz) | `ad346bbe2a053c1106b51e5125698737dc7b76fa3bf439e14d4b4ba1c262678fede9c507c1098aac6e14d2c742c526c8d257fefa95dd3bbb1dff959e1dc7b9aa` 875 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-server-linux-ppc64le.tar.gz) | `49f9bd1c751620ecf4b5c152f287d72b36abca21fd1dfe99443d984473c6efa051a910de585c42f5447ef7c18d7dbd905a66c4f09ca6025f45e63f5e96e3ca2f` 876 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-server-linux-s390x.tar.gz) | `d6be847f2a0358755a69dea26181e5fc1a80ac4939b8b04a3875e1f6693553cad562452bfad21b2e380ddda1839ab846122bc3339d8bec0971f218f6e8f6dce9` 877 878 ### Node Binaries 879 880 filename | sha512 hash 881 -------- | ----------- 882 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-node-linux-amd64.tar.gz) | `75dc99919d1084d7d471a53ab60c743dc399145c99e83f37c6ba3c241b2c0b2ecc2c0d1b94690ff912e2a15b7c5595aa1d2d24c2fc439e06d85ff0246fb43b89` 883 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-node-linux-arm.tar.gz) | `49013a4f01be8086fff332099d94903082688b9b295d2f34468462656da4709360025e9d84b069410c608977ef803079af09af1f1e2678af7cb64e0fc02e9c9d` 884 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-node-linux-arm64.tar.gz) | `f8c0cb0c089cd1d1977c049002620b8cf748d193c1b76dd1d3aac01ff9273549c06a1e3dfe983dc40a95ee8b0719908e0cdf86ce17359b5f1b2426f2c55799a1` 885 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-node-linux-ppc64le.tar.gz) | `48fc02c856a192388877189a43eb1cda531e548bb035f9dfe6a1e3c8d3bcbd0f8e14f29382da45702cb28a91126d13ede42bd6e9159e12ecbd387ca9a58f9a92` 886 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-node-linux-s390x.tar.gz) | `d7c5f52cf602fd0c0d0f72d4cfe1ceaa4bad70a42f37f21c103f17c3448ceb2396c1bfa521eeeb9eef5f3173d84e4268704a247edd826d765f65e9a29a4f7f72` 887 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0/kubernetes-node-windows-amd64.tar.gz) | `120afdebe844b06a7437bb9788c3e7ea4fc6352aa18cc6a00e70f44f54664f844429f138870bc15862579da632632dff2e7323be7f627d9c33585a11ad2bed6b` 888 889 # Kubernetes v1.14 Release Notes 890 891 ## 1.14 What’s New 892 893 Support for Windows Nodes is Graduating to Stable ([#116](https://github.com/kubernetes/enhancements/issues/116) ) 894 895 - Support for Windows Server 2019 for worker nodes and containers 896 - Support for out of tree networking with Azure-CNI, OVN-Kubernetes and Flannel 897 - Improved support for pods, service types, workload controllers and metrics/quotas to closely match the capabilities offered for Linux containers 898 kubernetes/enhancements: [#116](https://github.com/kubernetes/enhancements/issues/116) [[kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/116-windows-node-support/README.md)] 899 900 Updated Plugin Mechanism for kubectl is Graduating to Stable ([#579](https://github.com/kubernetes/enhancements/issues/579)) 901 902 - Extends functionality to kubectl to support extensions adding new commands as well as overriding specific subcommands (at any depth). 903 - Documentation fixes 904 kubernetes/enhancements: [#579](https://github.com/kubernetes/enhancements/issues/579) [[kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/0024-kubectl-plugins.md#summary)] 905 906 Durable Local Storage Management is Now GA ([#121](https://github.com/kubernetes/enhancements/issues/121#issuecomment-457396290)) 907 908 - Makes locally attached (non-network attached) storage available as a persistent volume source. 909 - Allows users to take advantage of the typically cheaper and improved performance of persistent local storage 910 kubernetes/kubernetes: [#73525](https://github.com/kubernetes/kubernetes/pull/73525), [#74391](https://github.com/kubernetes/kubernetes/pull/74391), [#74769](http://github.com/kubernetes/kubernetes/pull/74769) 911 kubernetes/enhancements: [#121](https://github.com/kubernetes/enhancements/issues/121#issuecomment-457396290) [[kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/121-local-persistent-volumes/README.md)] 912 913 Pid Limiting is Graduating to Beta ([#757](https://github.com/kubernetes/enhancements/issues/757)) 914 915 - Prevents a pod from starving pid resource 916 - Ability to isolate pid resources pod-to-pod and node-to-pod 917 kubernetes/kubernetes: [#73651](https://github.com/kubernetes/kubernetes/pull/73651) 918 kubernetes/enhancements: [#757](https://github.com/kubernetes/enhancements/issues/757) [[kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/757-pid-limiting/README.md)] 919 920 Pod Priority and Preemption in Kubernetes ([#564](https://github.com/kubernetes/enhancements/issues/564)) 921 922 - Pod priority and preemption enables Kubernetes scheduler to schedule more important Pods first and when cluster is out of resources, it removes less important pods to create room for more important ones. The importance is specified by priority. 923 kubernetes/kubernetes: [#73498](https://github.com/kubernetes/kubernetes/pull/73498), [#73555](https://github.com/kubernetes/kubernetes/pull/73555), [#74465](https://github.com/kubernetes/kubernetes/pull/74465) 924 kubernetes/enhancements: [#564](https://github.com/kubernetes/enhancements/issues/564) [[kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/268-priority-preemption/README.md)] 925 926 Pod Ready++ ([#580](https://github.com/kubernetes/enhancements/issues/580)) 927 928 - Introduces extension point for external feedback on pod readiness. 929 kubernetes/kubernetes: [#74434](http://github.com/kubernetes/kubernetes/pull/74434), 930 kubernetes/enhancements: [#580](https://github.com/kubernetes/enhancements/issues/580) [[kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/580-pod-readiness-gates/README.md)] 931 932 Kubeadm: Automate certificate copy between control planes in HA setups 933 934 - Joining control plane nodes to a HA cluster can now be simplified by enabling the optional automatic copy of certificates from an existing control plane node. 935 - You can now use `kubeadm init --experimental-upload-certs` and `kubeadm join --experimental-control-plane --certificate-key`. 936 kubernetes/kubeadm: [#1373](https://github.com/kubernetes/kubeadm/issues/1373) 937 kubernetes/enhancements: [#357](https://github.com/kubernetes/enhancements/issues/357) [kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/20190122-Certificates-copy-for-kubeadm-join--control-plane.md)] 938 939 Kubeadm: Expose the `kubeadm join` workflow as phases 940 941 - The `kubeadm join` command can now be used in phases. Similar to the work that was done for `kubeadm init` in 1.13, in 1.14 the `join` phases can be now executed step-by-step/selectively using the `kubeadm join phase` sub-command. This makes it possible to further customize the workflow of joining nodes to the cluster. 942 kubernetes/kubeadm: [#1204](https://github.com/kubernetes/kubeadm/issues/1204) 943 kubernetes/enhancements: [kep](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/2501-kubeadm-phases-to-beta/README.md) 944 945 ## Known Issues 946 947 - There is a known issue [coredns/coredns#2629](https://github.com/coredns/coredns/issues/2629) in CoreDNS 1.3.1, wherein if the Kubernetes API shuts down while CoreDNS is connected, CoreDNS will crash. The issue is fixed in CoreDNS 1.4.0 in [coredns/coredns#2529](https://github.com/coredns/coredns/pull/2529). 948 - Kubelet might fail to restart if an existing flexvolume mounted pvc contains a large number of directories, or is full. [#75019](https://github.com/kubernetes/kubernetes/pull/75019) 949 950 ## Urgent Upgrade Notes 951 952 ### (No, really, you MUST read this before you upgrade) 953 954 - kube-apiserver: 955 - Default RBAC policy no longer grants access to discovery and permission-checking APIs (used by `kubectl auth can-i`) to *unauthenticated* users. Upgraded clusters preserve prior behavior, but cluster administrators wishing to grant unauthenticated users access in new clusters will need to explicitly opt-in to expose the discovery and/or permission-checking APIs: ([#73807](https://github.com/kubernetes/kubernetes/pull/73807), [@dekkagaijin](https://github.com/dekkagaijin)) 956 - `kubectl create clusterrolebinding anonymous-discovery --clusterrole=system:discovery --group=system:unauthenticated` 957 - `kubectl create clusterrolebinding anonymous-access-review --clusterrole=system:basic-user --group=system:unauthenticated` 958 - The deprecated --storage-versions flag has been removed. The storage versions will always be the default value built-in the kube-apiserver binary. ([#67678](https://github.com/kubernetes/kubernetes/pull/67678), [@caesarxuchao](https://github.com/caesarxuchao)) 959 - The deprecated `--repair-malformed-updates` flag has been removed ([#73663](https://github.com/kubernetes/kubernetes/pull/73663), [@danielqsj](https://github.com/danielqsj)) 960 - The `/swaggerapi/*` schema docs, deprecated since 1.7, have been removed in favor of the /openapi/v2 schema docs. ([#72924](https://github.com/kubernetes/kubernetes/pull/72924), [@liggitt](https://github.com/liggitt)) 961 - The /swagger.json and /swagger-2.0.0.pb-v1 schema documents, deprecated since v1.10, have been removed in favor of `/openapi/v2` ([#73148](https://github.com/kubernetes/kubernetes/pull/73148), [@liggitt](https://github.com/liggitt)) 962 - `kube-apiserver` now only aggregates openapi schemas from `/openapi/v2` endpoints of aggregated API servers. The fallback to aggregate from `/swagger.json` has been removed. Ensure aggregated API servers provide schema information via `/openapi/v2` (available since v1.10). ([#73441](https://github.com/kubernetes/kubernetes/pull/73441), [@roycaihw](https://github.com/roycaihw)) 963 - The OpenAPI definitions with the prefix "io.k8s.kubernetes.pkg" (deprecated since 1.9) have been removed. ([#74596](https://github.com/kubernetes/kubernetes/pull/74596), [@sttts](https://github.com/sttts)) 964 - The `ValidateProxyRedirects` feature was promoted to Beta and enabled by default. This feature restricts redirect-following from the apiserver to same-host redirects. If nodes are configured to respond to CRI streaming requests on a different host interface than what the apiserver makes requests on (only the case if not using the built-in dockershim & setting the kubelet flag `--redirect-container-streaming=true`), then these requests will be broken. In that case, the feature can be temporarily disabled until the node configuration is corrected. We suggest setting `--redirect-container-streaming=false` on the kubelet to avoid issues.([#72552](https://github.com/kubernetes/kubernetes/pull/72552), [@tallclair](https://github.com/tallclair)) 965 966 - kubectl 967 - The deprecated `--show-all` flag to `kubectl get` has been removed ([#69255](https://github.com/kubernetes/kubernetes/pull/69255), [@Pingan2017](https://github.com/Pingan2017)) 968 969 - kubelet 970 - The deprecated `--experimental-fail-swap-on` flag has been removed ([#69552](https://github.com/kubernetes/kubernetes/pull/69552), [@Pingan2017](https://github.com/Pingan2017)) 971 - Health check (liveness & readiness) probes using an HTTPGetAction will no longer follow redirects to different hostnames from the original probe request. Instead, these non-local redirects will be treated as a Success (the documented behavior). In this case an event with reason "ProbeWarning" will be generated, indicating that the redirect was ignored. If you were previously relying on the redirect to run health checks against different endpoints, you will need to perform the healthcheck logic outside the Kubelet, for instance by proxying the external endpoint rather than redirecting to it. ([#75416](https://github.com/kubernetes/kubernetes/pull/75416), [@tallclair](https://github.com/tallclair)) 972 973 - client-go 974 - The deprecated versionless API group accessors (like `clientset.Apps()`) have been removed. Use an explicit version instead (like `clientset.AppsV1()`) ([#74422](https://github.com/kubernetes/kubernetes/pull/74422), [@liggitt](https://github.com/liggitt)) 975 - The disk-cached discovery client is moved from k8s.io/client-go/discovery to k8s.io/client-go/discovery/cached/disk. 976 The memory-cached discovery client is moved from k8s.io/client-go/discovery/cached to k8s.io/client-go/discovery/cached/memory. 977 ([#72214](https://github.com/kubernetes/kubernetes/pull/72214), [@caesarxuchao](https://github.com/caesarxuchao)) 978 979 - kubeadm 980 - `kubeadm alpha preflight` and `kubeadm alpha preflight node` are removed; you can now use `kubeadm join phase preflight` ([#73718](https://github.com/kubernetes/kubernetes/pull/73718), [@fabriziopandini](https://github.com/fabriziopandini)) 981 982 - The deprecated taints `node.alpha.kubernetes.io/notReady` and `node.alpha.kubernetes.io/unreachable` are no longer supported or adjusted. These uses should be replaced with `node.kubernetes.io/not-ready` and `node.kubernetes.io/unreachable` 983 ([#73001](https://github.com/kubernetes/kubernetes/pull/73001), [@shivnagarajan](https://github.com/shivnagarajan)) 984 985 - Any Prometheus queries that match `pod_name` and `container_name` labels (e.g. cadvisor or kubelet probe metrics) should be updated to use `pod` and `container` instead. `pod_name` and `container_name` labels will be present alongside `pod` and `container` labels for one transitional release and removed in the future. 986 ([#69099](https://github.com/kubernetes/kubernetes/pull/69099), [@ehashman](https://github.com/ehashman)) 987 988 ## Deprecations 989 990 - kubectl 991 - `kubectl convert` is deprecated and will be removed in v1.17. 992 - The `--export` flag for the `kubectl get` command is deprecated and will be removed in v1.18. ([#73787](https://github.com/kubernetes/kubernetes/pull/73787), [@soltysh](https://github.com/soltysh)) 993 994 - kubelet 995 - OS and Arch information is now recorded in `kubernetes.io/os` and `kubernetes.io/arch` labels on Node objects. The previous labels (`beta.kubernetes.io/os` and `beta.kubernetes.io/arch`) are still recorded, but are deprecated and targeted for removal in v1.18. ([#73333](https://github.com/kubernetes/kubernetes/pull/73333), [@yujuhong](https://github.com/yujuhong)) 996 - The `--containerized` flag is deprecated and will be removed in a future release ([#74267](https://github.com/kubernetes/kubernetes/pull/74267), [@dims](https://github.com/dims)) 997 998 - hyperkube 999 - The `--make-symlinks` flag is deprecated and will be removed in a future release. ([#74975](https://github.com/kubernetes/kubernetes/pull/74975), [@dims](https://github.com/dims)) 1000 - API 1001 - Ingress resources are now available via `networking.k8s.io/v1beta1`. Ingress resources in `extensions/v1beta1` are deprecated and will no longer be served in v1.18. Existing persisted data is available via the new API group/version ([#74057](https://github.com/kubernetes/kubernetes/pull/74057), [@liggitt](https://github.com/liggitt)) 1002 - NetworkPolicy resources will no longer be served from `extensions/v1beta1` in v1.16. Migrate use to the `networking.k8s.io/v1` API, available since v1.8. Existing persisted data can be retrieved via the `networking.k8s.io/v1` API. 1003 - PodSecurityPolicy resources will no longer be served from `extensions/v1beta1` in v1.16. Migrate to the `policy/v1beta1` API, available since v1.10. Existing persisted data can be retrieved via the `policy/v1beta1` API. 1004 - DaemonSet, Deployment, and ReplicaSet resources will no longer be served from `extensions/v1beta1`, `apps/v1beta1`, or `apps/v1beta2` in v1.16. Migrate to the `apps/v1` API, available since v1.9. Existing persisted data can be retrieved via the `apps/v1` API. 1005 - PriorityClass resources have been promoted to `scheduling.k8s.io/v1` with no changes. The `scheduling.k8s.io/v1beta1` and `scheduling.k8s.io/v1alpha1` versions are now deprecated and will stop being served by default in v1.17. ([#73555](https://github.com/kubernetes/kubernetes/pull/73555), [#74465](https://github.com/kubernetes/kubernetes/pull/74465), [@bsalamat](https://github.com/bsalamat)) 1006 - The `export` query parameter for list API calls is deprecated and will be removed in v1.18 ([#73783](https://github.com/kubernetes/kubernetes/pull/73783), [@deads2k](https://github.com/deads2k)) 1007 - The following features are now GA, and the associated feature gates are deprecated and will be removed in v1.15: 1008 - `CustomPodDNS` 1009 - `HugePages` 1010 - `MountPropagation` 1011 - `PersistentLocalVolumes` 1012 - CoreDNS: The following directives or keywords are deprecated and will be removed in v1.15: 1013 - `upstream` option of `kubernetes` plugin, becoming default behavior in v1.15. 1014 - `proxy` plugin replaced by `forward` plugin 1015 1016 ## Removed and deprecated metrics 1017 1018 ### Removed metrics 1019 1020 - `reflector_items_per_list` 1021 - `reflector_items_per_watch` 1022 - `reflector_last_resource_version` 1023 - `reflector_list_duration_seconds` 1024 - `reflector_lists_total` 1025 - `reflector_short_watches_total` 1026 - `reflector_watch_duration_seconds` 1027 - `reflector_watches_total` 1028 1029 ### Deprecated metrics 1030 1031 - `rest_client_request_latency_seconds` -> `rest_client_request_duration_seconds` 1032 - `apiserver_proxy_tunnel_sync_latency_secs` -> `apiserver_proxy_tunnel_sync_duration_seconds` 1033 - `scheduler_scheduling_latency_seconds` -> `scheduler_scheduling_duration_seconds` 1034 - `kubelet_pod_worker_latency_microseconds` -> `kubelet_pod_worker_duration_seconds` 1035 - `kubelet_pod_start_latency_microseconds` -> `kubelet_pod_start_duration_seconds` 1036 - `kubelet_cgroup_manager_latency_microseconds` -> `kubelet_cgroup_manager_duration_seconds` 1037 - `kubelet_pod_worker_start_latency_microseconds` -> `kubelet_pod_worker_start_duration_seconds` 1038 - `kubelet_pleg_relist_latency_microseconds` -> `kubelet_pleg_relist_duration_seconds` 1039 - `kubelet_pleg_relist_interval_microseconds` -> `kubelet_pleg_relist_interval_seconds` 1040 - `kubelet_eviction_stats_age_microseconds` -> `kubelet_eviction_stats_age_seconds` 1041 - `kubelet_runtime_operations` -> `kubelet_runtime_operations_total` 1042 - `kubelet_runtime_operations_latency_microseconds` -> `kubelet_runtime_operations_duration_seconds` 1043 - `kubelet_runtime_operations_errors` -> `kubelet_runtime_operations_errors_total` 1044 - `kubelet_device_plugin_registration_count` -> `kubelet_device_plugin_registration_total` 1045 - `kubelet_device_plugin_alloc_latency_microseconds` -> `kubelet_device_plugin_alloc_duration_seconds` 1046 - `docker_operations` -> `docker_operations_total` 1047 - `docker_operations_latency_microseconds` -> `docker_operations_latency_seconds` 1048 - `docker_operations_errors` -> `docker_operations_errors_total` 1049 - `docker_operations_timeout` -> `docker_operations_timeout_total` 1050 - `network_plugin_operations_latency_microseconds` -> `network_plugin_operations_latency_seconds` 1051 - `sync_proxy_rules_latency_microseconds` -> `sync_proxy_rules_latency_seconds` 1052 - `apiserver_request_count` -> `apiserver_request_total` 1053 - `apiserver_request_latencies` -> `apiserver_request_latency_seconds` 1054 - `apiserver_request_latencies_summary` -> `apiserver_request_latency_seconds` 1055 - `apiserver_dropped_requests` -> `apiserver_dropped_requests_total` 1056 - `etcd_helper_cache_hit_count` -> `etcd_helper_cache_hit_total` 1057 - `etcd_helper_cache_miss_count` -> `etcd_helper_cache_miss_total` 1058 - `etcd_helper_cache_entry_count` -> `etcd_helper_cache_entry_total` 1059 - `etcd_request_cache_get_latencies_summary` -> `etcd_request_cache_get_latency_seconds` 1060 - `etcd_request_cache_add_latencies_summary` -> `etcd_request_cache_add_latency_seconds` 1061 - `etcd_request_latencies_summary` -> `etcd_request_latency_seconds` 1062 - `transformation_latencies_microseconds` -> `transformation_latencies_seconds` 1063 - `data_key_generation_latencies_microseconds` -> `data_key_generation_latencies_seconds` 1064 1065 ## Notable Features 1066 1067 - Increased the histogram resolution of the API server client certificate to accommodate short-lived (< 6h) client certificates. ([#74806](https://github.com/kubernetes/kubernetes/pull/74806), [@mxinden](https://github.com/mxinden)) 1068 - Updated to use golang 1.12 ([#74632](https://github.com/kubernetes/kubernetes/pull/74632), [@cblecker](https://github.com/cblecker)) 1069 - The `RunAsGroup` feature has been promoted to beta and enabled by default. `PodSpec` and `PodSecurityPolicy` objects can be used to control the primary GID of containers on supported container runtimes. ([#73007](https://github.com/kubernetes/kubernetes/pull/73007), [@krmayankk](https://github.com/krmayankk)) 1070 - Added the same information to an init container as a standard container in a pod when using `PodPresets`. ([#71479](https://github.com/kubernetes/kubernetes/pull/71479), [@soggiest](https://github.com/soggiest)) 1071 - kube-conformance image will now run ginkgo with the `--dryRun` flag if the container is run with the environment variable E2E_DRYRUN set. ([#74731](https://github.com/kubernetes/kubernetes/pull/74731), [@johnSchnake](https://github.com/johnSchnake)) 1072 - Introduced dynamic volume provisioning shim for CSI migration ([#73653](https://github.com/kubernetes/kubernetes/pull/73653), [@ddebroy](https://github.com/ddebroy)) 1073 - Applied resources from a directory containing kustomization.yaml ([#74140](https://github.com/kubernetes/kubernetes/pull/74140), [@Liujingfang1](https://github.com/Liujingfang1)) 1074 - kubeadm: Allowed to download certificate secrets uploaded by `init` or `upload-certs` phase, allowing to transfer certificate secrets (certificates and keys) from the cluster to other master machines when creating HA deployments. ([#74168](https://github.com/kubernetes/kubernetes/pull/74168), [@ereslibre](https://github.com/ereslibre)) 1075 - The `--quiet` option to `kubectl run` now suppresses resource deletion messages emitted when the `--rm` option is specified. ([#73266](https://github.com/kubernetes/kubernetes/pull/73266), [@awh](https://github.com/awh)) 1076 - Added Custom Resource support to `kubectl autoscale` ([#72678](https://github.com/kubernetes/kubernetes/pull/72678), [@rmohr](https://github.com/rmohr)) 1077 - Cinder volume limit can now be configured from node too ([#74542](https://github.com/kubernetes/kubernetes/pull/74542), [@gnufied](https://github.com/gnufied)) 1078 - It is now possible to combine the `-f` and `-l` flags in `kubectl logs` ([#67573](https://github.com/kubernetes/kubernetes/pull/67573), [@m1kola](https://github.com/m1kola)) 1079 - New conformance tests added for API Aggregation. ([#63947](https://github.com/kubernetes/kubernetes/pull/63947), [@jennybuckley](https://github.com/jennybuckley)) 1080 - Moved fluentd-elasticsearch addon images to community controlled location ([#73819](https://github.com/kubernetes/kubernetes/pull/73819), [@coffeepac](https://github.com/coffeepac)) 1081 - Removed local etcd members from the etcd cluster when `kubeadm reset` ([#74112](https://github.com/kubernetes/kubernetes/pull/74112), [@pytimer](https://github.com/pytimer)) 1082 - kubeadm will now not fail preflight checks when running on >= 5.0 Linux kernel ([#74355](https://github.com/kubernetes/kubernetes/pull/74355), [@brb](https://github.com/brb)) 1083 - Scheduler cache snapshot algorithm has been optimized to improve scheduling throughput. ([#74041](https://github.com/kubernetes/kubernetes/pull/74041), [@bsalamat](https://github.com/bsalamat)) 1084 - It is now possible to upload certificates required to join a new control-plane to kubeadm-certs secret using the flag `--experimental-upload-certs` on `init` or upload-certs phase. ([#73907](https://github.com/kubernetes/kubernetes/pull/73907), [@yagonobre](https://github.com/yagonobre)) 1085 - `kubectl auth reconcile` now outputs details about what changes are being made ([#71564](https://github.com/kubernetes/kubernetes/pull/71564), [@liggitt](https://github.com/liggitt)) 1086 - Added Kustomize as a subcommand in kubectl ([#73033](https://github.com/kubernetes/kubernetes/pull/73033), [@Liujingfang1](https://github.com/Liujingfang1)) 1087 - Added `kubelet_node_name` metrics. ([#72910](https://github.com/kubernetes/kubernetes/pull/72910), [@danielqsj](https://github.com/danielqsj)) 1088 - Updated AWS SDK to v1.16.26 for ECR PrivateLink support ([#73435](https://github.com/kubernetes/kubernetes/pull/73435), [@micahhausler](https://github.com/micahhausler)) 1089 - Expanded `kubectl wait` to work with more types of selectors. ([#71746](https://github.com/kubernetes/kubernetes/pull/71746), [@rctl](https://github.com/rctl)) 1090 - Added configuration for AWS endpoint fine control: ([#72245](https://github.com/kubernetes/kubernetes/pull/72245), [@ampsingram](https://github.com/ampsingram)) 1091 - The CoreDNS configuration now has the forward plugin for proxy in the default configuration instead of the proxy plugin. ([#73267](https://github.com/kubernetes/kubernetes/pull/73267), [@rajansandeep](https://github.com/rajansandeep)) 1092 - Added alpha field storageVersionHash to the discovery document for each resource. Its value must be treated as opaque by clients. Only equality comparison on the value is valid. ([#73191](https://github.com/kubernetes/kubernetes/pull/73191), [@caesarxuchao](https://github.com/caesarxuchao)) 1093 - If you are running the cloud-controller-manager and you have the `pvlabel.kubernetes.io` alpha Initializer enabled, you must now enable PersistentVolume labeling using the `PersistentVolumeLabel` admission controller instead. You can do this by adding `PersistentVolumeLabel` in the `--enable-admission-plugins` kube-apiserver flag. ([#73102](https://github.com/kubernetes/kubernetes/pull/73102), [@andrewsykim](https://github.com/andrewsykim)) 1094 - kubectl supports copying files with wild card ([#72641](https://github.com/kubernetes/kubernetes/pull/72641), [@dixudx](https://github.com/dixudx)) 1095 - kubeadm now attempts to detect an installed CRI by its usual domain socket, so that `--cri-socket` can be omitted from the command line if Docker is not used and there is a single CRI installed. ([#69366](https://github.com/kubernetes/kubernetes/pull/69366), [@rosti](https://github.com/rosti)) 1096 - `CSINodeInfo` and `CSIDriver` CRDs have been installed in the local cluster. ([#72584](https://github.com/kubernetes/kubernetes/pull/72584), [@xing-yang](https://github.com/xing-yang)) 1097 - Node OS/arch labels have been promoted to GA ([#73048](https://github.com/kubernetes/kubernetes/pull/73048), [@yujuhong](https://github.com/yujuhong)) 1098 - Added support for max attach limit for Cinder ([#72980](https://github.com/kubernetes/kubernetes/pull/72980), [@gnufied](https://github.com/gnufied)) 1099 - Enabled mTLS encryption between etcd and kube-apiserver in GCE ([#70144](https://github.com/kubernetes/kubernetes/pull/70144), [@wenjiaswe](https://github.com/wenjiaswe)) 1100 - Added `ResourceVersion` as a precondition for delete in order to ensure a delete fails if an unobserved change happens to an object. ([#74040](https://github.com/kubernetes/kubernetes/pull/74040), [@ajatprabha](https://github.com/ajatprabha)) 1101 - There is now support for collecting pod logs under `/var/log/pods/NAMESPACE_NAME_UID` to stackdriver with `k8s_pod` resource type. ([#74502](https://github.com/kubernetes/kubernetes/pull/74502), [@Random-Liu](https://github.com/Random-Liu)) 1102 - Changed CRI pod log directory from `/var/log/pods/UID` to `/var/log/pods/NAMESPACE_NAME_UID`. ([#74441](https://github.com/kubernetes/kubernetes/pull/74441), [@Random-Liu](https://github.com/Random-Liu)) 1103 - `RuntimeClass` has been promoted to beta, and is enabled by default. ([#75003](https://github.com/kubernetes/kubernetes/pull/75003), [@tallclair](https://github.com/tallclair)) 1104 - New "dry_run" metric label (indicating the value of the dryRun query parameter) has been added into the metrics: 1105 * apiserver_request_total 1106 * apiserver_request_duration_seconds 1107 New "APPLY" value for the "verb" metric label which indicates a PATCH with "Content-Type: apply-patch+yaml". This value is experimental and will only be present if the ServerSideApply alpha feature is enabled. ([#74997](https://github.com/kubernetes/kubernetes/pull/74997), [@jennybuckley](https://github.com/jennybuckley)) 1108 - GCE: bumped COS image version to `cos-beta-73-11647-64-0` ([#75149](https://github.com/kubernetes/kubernetes/pull/75149), [@yguo0905](https://github.com/yguo0905)) 1109 - Added alpha support for ephemeral CSI inline volumes that are embedded in pod specs. ([#74086](https://github.com/kubernetes/kubernetes/pull/74086), [@vladimirvivien](https://github.com/vladimirvivien)) 1110 - The NodeLease feature gate has been promoted to Beta, and is enabled by default ([#72096](https://github.com/kubernetes/kubernetes/pull/72096), [@wojtek-t](https://github.com/wojtek-t)) 1111 1112 ## API Changes 1113 1114 - [CRI] Added a new field called `runtime_handler` into `PodSandbox` and `PodSandboxStatus` to track the `RuntimeClass` information of a pod. ([#73833](https://github.com/kubernetes/kubernetes/pull/73833), [@haiyanmeng](https://github.com/haiyanmeng)) 1115 1116 ## Detailed Bug Fixes And Changes 1117 1118 ### API Machinery 1119 1120 - client-go: `PortForwarder.GetPorts()` now contain correct local port if no local port was initially specified when setting up the port forwarder ([#73676](https://github.com/kubernetes/kubernetes/pull/73676), [@martin-helmich](https://github.com/martin-helmich)) 1121 - Fixed an issue with missing `apiVersion/kind` in object data sent to admission webhooks ([#74448](https://github.com/kubernetes/kubernetes/pull/74448), [@liggitt](https://github.com/liggitt)) 1122 - Prometheus metrics for `crd_autoregister`, `crd_finalizer` and `crd_naming_condition_controller` are exported. ([#71767](https://github.com/kubernetes/kubernetes/pull/71767), [@roycaihw](https://github.com/roycaihw)) 1123 - Fixed admission metrics in seconds. ([#72343](https://github.com/kubernetes/kubernetes/pull/72343), [@danielqsj](https://github.com/danielqsj)) 1124 - When a watch is closed by an HTTP2 load balancer and we are told to go away, skip printing the message to stderr by default. 1125 - Spedup kubectl by >10 when calling out to kube-apiserver for discovery information. ([#73345](https://github.com/kubernetes/kubernetes/pull/73345), [@sttts](https://github.com/sttts)) 1126 - Fixed watch to not send the same set of events multiple times causing watcher to go back in time ([#73845](https://github.com/kubernetes/kubernetes/pull/73845), [@wojtek-t](https://github.com/wojtek-t)) 1127 - Fix kube-apiserver not to create default/kubernetes service endpoints before it reports readiness via the /healthz and therefore is ready to serve requests. Also early during startup old endpoints are remove which might be left over from a previously crashed kube-apiserver. ([#74668](https://github.com/kubernetes/kubernetes/pull/74668), [@sttts](https://github.com/sttts)) 1128 - Add a configuration field to shorten the timeout of validating/mutating admission webhook call. The timeout value must be between 1 and 30 seconds. Default to 30 seconds when unspecified. ([#74562](https://github.com/kubernetes/kubernetes/pull/74562), [@roycaihw](https://github.com/roycaihw)) 1129 - The apiserver, including both the kube-apiserver and apiservers built with the generic apiserver library, will now return 413 RequestEntityTooLarge error if a json patch contains more than 10,000 operations. ([#74000](https://github.com/kubernetes/kubernetes/pull/74000), [@caesarxuchao](https://github.com/caesarxuchao)) 1130 - Fixed an error processing watch events when running skewed apiservers ([#73482](https://github.com/kubernetes/kubernetes/pull/73482), [@liggitt](https://github.com/liggitt)) 1131 - jsonpath expressions containing `[start:end:step]` slice are now evaluated correctly ([#73149](https://github.com/kubernetes/kubernetes/pull/73149), [@liggitt](https://github.com/liggitt)) 1132 - `metadata.deletionTimestamp` is no longer moved into the future when issuing repeated DELETE requests against a resource containing a finalizer. ([#73138](https://github.com/kubernetes/kubernetes/pull/73138), [@liggitt](https://github.com/liggitt)) 1133 - Fixed kube-apiserver not to create default/kubernetes service endpoints before it reports readiness via the /healthz and therefore is ready to serve requests. Also early during startup old endpoints are remove which might be left over from a previously crashed kube-apiserver. ([#74668](https://github.com/kubernetes/kubernetes/pull/74668), [@sttts](https://github.com/sttts)) 1134 - `watch.Until` now works for long durations. ([#67350](https://github.com/kubernetes/kubernetes/pull/67350), [@tnozicka](https://github.com/tnozicka)) 1135 - Added duration metric for CRD webhook converters. ([#74376](https://github.com/kubernetes/kubernetes/pull/74376), [@mbohlool](https://github.com/mbohlool)) 1136 - Fixed keymutex issues which may crash in some platforms. ([#74348](https://github.com/kubernetes/kubernetes/pull/74348), [@danielqsj](https://github.com/danielqsj)) 1137 - Considerably reduced the CPU load in kube-apiserver while aggregating OpenAPI specifications from aggregated API servers. ([#71223](https://github.com/kubernetes/kubernetes/pull/71223), [@sttts](https://github.com/sttts)) 1138 - Fixed graceful apiserver shutdown to not drop outgoing bytes before the process terminates. ([#72970](https://github.com/kubernetes/kubernetes/pull/72970), [@sttts](https://github.com/sttts)) 1139 1140 ### Apps 1141 1142 - Added deleting pods created by `DaemonSet` assigned to not existing nodes. ([#73401](https://github.com/kubernetes/kubernetes/pull/73401), [@krzysztof-jastrzebski](https://github.com/krzysztof-jastrzebski)) 1143 - Pod eviction now honors graceful deletion by default if no delete options are provided in the eviction request. ([#72730](https://github.com/kubernetes/kubernetes/pull/72730), [@liggitt](https://github.com/liggitt)) 1144 1145 ### Auth 1146 1147 - Added `kubectl auth can-i --list` option, which allows users to know what actions they can do in specific namespaces. ([#64820](https://github.com/kubernetes/kubernetes/pull/64820), [@WanLinghao](https://github.com/WanLinghao)) 1148 - The `rules` field in RBAC `Role` and `ClusterRole` objects is now correctly reported as optional in the openapi schema. ([#73250](https://github.com/kubernetes/kubernetes/pull/73250), [@liggitt](https://github.com/liggitt)) 1149 - `system:kube-controller-manager` and `system:kube-scheduler` users are now permitted to perform delegated authentication/authorization checks by default RBAC policy ([#72491](https://github.com/kubernetes/kubernetes/pull/72491), [@liggitt](https://github.com/liggitt)) 1150 - Error messages returned in authentication webhook status responses are now correctly included in the apiserver log ([#73595](https://github.com/kubernetes/kubernetes/pull/73595), [@liggitt](https://github.com/liggitt)) 1151 - Fixed use of webhook admission plugins with multi-version custom resources ([#74154](https://github.com/kubernetes/kubernetes/pull/74154), [@mbohlool](https://github.com/mbohlool)) 1152 1153 ### AWS 1154 1155 - Prevented AWS Network Load Balancer security groups ingress rules to be deleted by ensuring target groups are tagged. ([#73594](https://github.com/kubernetes/kubernetes/pull/73594), [@masterzen](https://github.com/masterzen)) 1156 - AWS ELB health checks will now use HTTPS/SSL protocol for HTTPS/SSL backends. ([#70309](https://github.com/kubernetes/kubernetes/pull/70309), [@2rs2ts](https://github.com/2rs2ts)) 1157 1158 ### Azure 1159 1160 - Fixed failure to detach Azure disk when there is server side error ([#74398](https://github.com/kubernetes/kubernetes/pull/74398), [@andyzhangx](https://github.com/andyzhangx)) 1161 - Fixed subnet annotation checking for Azure internal loadbalancer ([#74498](https://github.com/kubernetes/kubernetes/pull/74498), [@feiskyer](https://github.com/feiskyer)) 1162 - Fixed mixed protocol issue for Azure load balancer ([#74200](https://github.com/kubernetes/kubernetes/pull/74200), [@andyzhangx](https://github.com/andyzhangx)) 1163 - Fixed Azure accounts timeout issue when there is no out-bound IP ([#74191](https://github.com/kubernetes/kubernetes/pull/74191), [@andyzhangx](https://github.com/andyzhangx)) 1164 - Fixed Azure Container Registry anonymous repo image pull error ([#74715](https://github.com/kubernetes/kubernetes/pull/74715), [@andyzhangx](https://github.com/andyzhangx)) 1165 - Fixed parse devicePath issue on Azure Disk ([#74499](https://github.com/kubernetes/kubernetes/pull/74499), [@andyzhangx](https://github.com/andyzhangx)) 1166 1167 ### CLI 1168 1169 - Fixed `--help` flag parsing ([#74682](https://github.com/kubernetes/kubernetes/pull/74682), [@soltysh](https://github.com/soltysh)) 1170 - Fixed a bug where `kubectl describe` cannot obtain the event messages for a static pod ([#74156](https://github.com/kubernetes/kubernetes/pull/74156), [@gaorong](https://github.com/gaorong)) 1171 - Fixed panic when performing a `set env` operation on a `--local` resource ([#65636](https://github.com/kubernetes/kubernetes/pull/65636), [@juanvallejo](https://github.com/juanvallejo)) 1172 - Missing directories listed in a user's PATH are no longer considered errors and are instead logged by the `kubectl plugin list` command when listing available plugins. ([#73542](https://github.com/kubernetes/kubernetes/pull/73542), [@juanvallejo](https://github.com/juanvallejo)) 1173 - Now users can get object info like: 1174 1175 ```bash 1176 a. kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[0:3].name 1177 b. kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[-2:].name 1178 ``` 1179 1180 ([#73063](https://github.com/kubernetes/kubernetes/pull/73063), [@WanLinghao](https://github.com/WanLinghao)) 1181 1182 - The `kubectl api-resources` command will no longer fail to display any resources on a single failure ([#73035](https://github.com/kubernetes/kubernetes/pull/73035), [@juanvallejo](https://github.com/juanvallejo)) 1183 - kubectl now loads config file once and uses persistent client config ([#71117](https://github.com/kubernetes/kubernetes/pull/71117), [@dixudx](https://github.com/dixudx)) 1184 - Printed `SizeLimit` of `EmptyDir` in `kubectl describe pod` outputs. ([#69279](https://github.com/kubernetes/kubernetes/pull/69279), [@dtaniwaki](https://github.com/dtaniwaki)) 1185 - `kubectl delete --all-namespaces` is now a recognized flag. ([#73716](https://github.com/kubernetes/kubernetes/pull/73716), [@deads2k](https://github.com/deads2k)) 1186 1187 ### Cloud Provider 1188 1189 - Fixed a bug that caused PV allocation on non-English vSphere installations to fail ([#73115](https://github.com/kubernetes/kubernetes/pull/73115), [@alvaroaleman](https://github.com/alvaroaleman)) 1190 1191 ### Cluster Lifecycle 1192 1193 - kubeadm: fixed nil pointer dereference caused by a bug in url parsing ([#74454](https://github.com/kubernetes/kubernetes/pull/74454), [@bart0sh](https://github.com/bart0sh)) 1194 - CoreDNS adds readinessProbe which prevents loadbalancing to unready pods, and also allows rolling updates to work as expected. ([#74137](https://github.com/kubernetes/kubernetes/pull/74137), [@rajansandeep](https://github.com/rajansandeep)) 1195 - kubeadm no longer allows using v1alpha3 configs for anything else than converting them to `v1beta1`. ([#74025](https://github.com/kubernetes/kubernetes/pull/74025), [@rosti](https://github.com/rosti)) 1196 - kubeadm: now allows the usage of `--kubeconfig-dir` and `--config` flags on kubeadm init ([#73998](https://github.com/kubernetes/kubernetes/pull/73998), [@yagonobre](https://github.com/yagonobre)) 1197 - kubeadm: all master components are now exclusively relying on the `PriorityClassName` pod spec for annotating them as cluster critical components. Since `scheduler.alpha.kubernetes.io/critical-pod` annotation is no longer supported by Kubernetes 1.14 this annotation is no longer added to master components. ([#73857](https://github.com/kubernetes/kubernetes/pull/73857), [@ereslibre](https://github.com/ereslibre)) 1198 - kubeadm no longer dumps backtrace if it fails to remove the running containers on reset. ([#73951](https://github.com/kubernetes/kubernetes/pull/73951), [@rosti](https://github.com/rosti)) 1199 - kubeadm: fixed a bug in the underlying library for diff related to characters like '%' ([#73941](https://github.com/kubernetes/kubernetes/pull/73941), [@neolit123](https://github.com/neolit123)) 1200 - Scale max-inflight now limits together with master VM sizes. ([#73268](https://github.com/kubernetes/kubernetes/pull/73268), [@wojtek-t](https://github.com/wojtek-t)) 1201 - kubeadm reset: fixed a crash caused by the absence of a configuration file ([#73636](https://github.com/kubernetes/kubernetes/pull/73636), [@bart0sh](https://github.com/bart0sh)) 1202 - CoreDNS is now version 1.3.1 ([#73610](https://github.com/kubernetes/kubernetes/pull/73610), [@rajansandeep](https://github.com/rajansandeep)) 1203 - kubeadm: When certificates are present in joining a new control plane now ensures that they match at least the required SANs ([#73093](https://github.com/kubernetes/kubernetes/pull/73093), [@ereslibre](https://github.com/ereslibre)) 1204 - kubeadm: added back `--cert-dir` option for `kubeadm init phase certs sa` ([#73239](https://github.com/kubernetes/kubernetes/pull/73239), [@mattkelly](https://github.com/mattkelly)) 1205 - kubeadm: now explicitly waits for `etcd` to have grown when joining a new control plane ([#72984](https://github.com/kubernetes/kubernetes/pull/72984), [@ereslibre](https://github.com/ereslibre)) 1206 - kubeadm: now pulls images when joining a new control plane instance ([#72870](https://github.com/kubernetes/kubernetes/pull/72870), [@MalloZup](https://github.com/MalloZup)) 1207 - Exited kube-proxy when configuration file changes ([#59176](https://github.com/kubernetes/kubernetes/pull/59176), [@dixudx](https://github.com/dixudx)) 1208 - kube-addon-manager was updated to v9.0, and now uses kubectl v1.13.2 and prunes workload resources via the apps/v1 API ([#72978](https://github.com/kubernetes/kubernetes/pull/72978), [@liggitt](https://github.com/liggitt)) 1209 - kubeadm: Now allows certain certs/keys to be missing on the secret when transferring secrets using `--experimental-upload-certs` feature ([#75415](https://github.com/kubernetes/kubernetes/pull/75415), [@ereslibre](https://github.com/ereslibre)) 1210 1211 ### GCP 1212 1213 - Fixed liveness probe in fluentd-gcp cluster addon ([#74522](https://github.com/kubernetes/kubernetes/pull/74522), [@Pluies](https://github.com/Pluies)) 1214 - Reduced GCE log rotation check from 1 hour to every 5 minutes. Rotation policy is unchanged (new day starts, log file size > 100MB). ([#72062](https://github.com/kubernetes/kubernetes/pull/72062), [@jpbetz](https://github.com/jpbetz)) 1215 1216 ### Network 1217 1218 - Reduces the cache TTL for negative responses to 5s minimum. ([#74093](https://github.com/kubernetes/kubernetes/pull/74093), [@blakebarnett](https://github.com/blakebarnett)) 1219 1220 ### Node 1221 1222 - Fixed help message for `--container-runtime-endpoint`: only unix socket is support on Linux. ([#74712](https://github.com/kubernetes/kubernetes/pull/74712), [@feiskyer](https://github.com/feiskyer)) 1223 - Image garbage collection no longer fails for images with only one tag but more than one repository associated. ([#70647](https://github.com/kubernetes/kubernetes/pull/70647), [@corvus-ch](https://github.com/corvus-ch)) 1224 - Re-issued Allocate grpc calls before starting a container that requests device-plugin resources if the cached state is missing. ([#73824](https://github.com/kubernetes/kubernetes/pull/73824), [@jiayingz](https://github.com/jiayingz)) 1225 - [CRI] Added a new field called `runtime_handler` into `PodSandbox` and `PodSandboxStatus` to track the `RuntimeClass` information of a pod. ([#73833](https://github.com/kubernetes/kubernetes/pull/73833), [@haiyanmeng](https://github.com/haiyanmeng)) 1226 - Kubelet now tries to stop containers in unknown state once before restart or remove. ([#73802](https://github.com/kubernetes/kubernetes/pull/73802), [@Random-Liu](https://github.com/Random-Liu)) 1227 - When pleg channel is full, events are now discarded and count is recorded ([#72709](https://github.com/kubernetes/kubernetes/pull/72709), [@changyaowei](https://github.com/changyaowei)) 1228 - Fixed the unexpected `NotReady` status when Node's iops is full if the runtime is dockershim. ([#74389](https://github.com/kubernetes/kubernetes/pull/74389), [@answer1991](https://github.com/answer1991)) 1229 - Fixed #73264 `cpuPeriod` was not reset, but used as set via flag, although it was disabled via alpha gate ([#73342](https://github.com/kubernetes/kubernetes/pull/73342), [@szuecs](https://github.com/szuecs)) 1230 - Updated kubelet CLI summary documentation and generated webpage ([#73256](https://github.com/kubernetes/kubernetes/pull/73256), [@deitch](https://github.com/deitch)) 1231 - Set a low `oom_score_adj` for containers in pods with system-critical priorities ([#73758](https://github.com/kubernetes/kubernetes/pull/73758), [@sjenning](https://github.com/sjenning)) 1232 - kubelet: Resolved hang/timeout issues when running large numbers of pods with unique `ConfigMap/Secret` references ([#74755](https://github.com/kubernetes/kubernetes/pull/74755), [@liggitt](https://github.com/liggitt)) 1233 - Events reported for container creation, start, and stop now report the container name in the message and are more consistently formatted. ([#73892](https://github.com/kubernetes/kubernetes/pull/73892), [@smarterclayton](https://github.com/smarterclayton)) 1234 - Removed stale `OutOfDisk` condition from kubelet side ([#72507](https://github.com/kubernetes/kubernetes/pull/72507), [@dixudx](https://github.com/dixudx)) 1235 - Fixed the setting of `NodeAddresses` when using the vSphere CloudProvider and nodes that have multiple IP addresses. ([#70805](https://github.com/kubernetes/kubernetes/pull/70805), [@danwinship](https://github.com/danwinship)) 1236 - Fixed dockershim panic issues when deleting docker images. ([#75367](https://github.com/kubernetes/kubernetes/pull/75367), [@feiskyer](https://github.com/feiskyer)) 1237 - Kubelet no longer watches `ConfigMaps` and `Secrets` for terminated pods, in worst scenario causing it to not be able to send other requests to kube-apiserver ([#74809](https://github.com/kubernetes/kubernetes/pull/74809), [@oxddr](https://github.com/oxddr)) 1238 - A new `TaintNodesByCondition` admission plugin taints newly created Node objects as "not ready", to fix a race condition that could cause pods to be scheduled on new nodes before their taints were updated to accurately reflect their reported conditions. This admission plugin is enabled by default if the `TaintNodesByCondition` feature is enabled. ([#73097](https://github.com/kubernetes/kubernetes/pull/73097), [@bsalamat](https://github.com/bsalamat)) 1239 - kubelet now accepts `pid=<number>` in the `--system-reserved` and `--kube-reserved` options to ensure that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively. Please reference `Kube Reserved` and `System Reserved` in `Reserve Compute Resources for System Daemons` in the Kubernetes documentation for general discussion of resource reservation. To utilize this functionality, you must set the feature gate `SupportNodePidsLimit=true` ([#73651](https://github.com/kubernetes/kubernetes/pull/73651), [@RobertKrawitz](https://github.com/RobertKrawitz)) 1240 1241 ### Scheduling 1242 1243 - Improved fairness of the scheduling queue by placing pods which are attempted recently behind other pods with the same priority. ([#73700](https://github.com/kubernetes/kubernetes/pull/73700), [@denkensk](https://github.com/denkensk)) 1244 - Improved scheduler robustness to ensure that unschedulable pods are reconsidered for scheduling when appropriate. ([#73700](https://github.com/kubernetes/kubernetes/pull/73700), [#72558](https://github.com/kubernetes/kubernetes/pull/72558), [@denkensk](https://github.com/denkensk), [#73078](https://github.com/kubernetes/kubernetes/pull/73078), [@Huang-Wei](https://github.com/Huang-Wei)) 1245 1246 ### Storage 1247 1248 - Fixed scanning of failed iSCSI targets. ([#74306](https://github.com/kubernetes/kubernetes/pull/74306), [@jsafrane](https://github.com/jsafrane)) 1249 - StorageOS volume plugin updated to fix an issue where volume mount succeeds even if request to mount via StorageOS API fails. ([#69782](https://github.com/kubernetes/kubernetes/pull/69782), [@darkowlzz](https://github.com/darkowlzz)) 1250 - Ensured directories on volumes are group-executable when using `fsGroup` ([#73533](https://github.com/kubernetes/kubernetes/pull/73533), [@mxey](https://github.com/mxey)) 1251 - Updated CSI version to 1.1 ([#75391](https://github.com/kubernetes/kubernetes/pull/75391), [@gnufied](https://github.com/gnufied)) 1252 - Ensured that volumes get provisioned based on the zone information provided in `allowedTopologies`. ([#72731](https://github.com/kubernetes/kubernetes/pull/72731), [@skarthiksrinivas](https://github.com/skarthiksrinivas)) 1253 - Extended the `VolumeSubpathEnvExpansion` alpha feature to support environment variable expansion ([#71351](https://github.com/kubernetes/kubernetes/pull/71351), [@kevtaylor](https://github.com/kevtaylor)) 1254 - Fixed a bug that prevented deletion of dynamically provisioned volumes in Quobyte backends. ([#68925](https://github.com/kubernetes/kubernetes/pull/68925), [@casusbelli](https://github.com/casusbelli)) 1255 1256 ### Testing 1257 1258 - e2e storage tests now run faster and are easier to read ([#72434](https://github.com/kubernetes/kubernetes/pull/72434), [@pohly](https://github.com/pohly)) 1259 - `e2e.test` now rejects unknown `--provider` values instead of merely warning about them. An empty provider name is not accepted anymore and was replaced by `skeleton` (a provider with no special behavior). ([#73402](https://github.com/kubernetes/kubernetes/pull/73402), [@pohly](https://github.com/pohly)) 1260 - Updated to go1.11.5 ([#73326](https://github.com/kubernetes/kubernetes/pull/73326), [@ixdy](https://github.com/ixdy)) 1261 - Updated to use go1.12.1 ([#75413](https://github.com/kubernetes/kubernetes/pull/75413), [@BenTheElder](https://github.com/BenTheElder)) 1262 - e2e tests that require SSH may now be used against clusters that have nodes without external IP addresses by setting the environment variable `KUBE_SSH_BASTION` to the `host:port` of a machine that is allowed to SSH to those nodes. The same private key that the test would use is used for the bastion host. The test connects to the bastion and then tunnels another SSH connection to the node. ([#72286](https://github.com/kubernetes/kubernetes/pull/72286), [@smarterclayton](https://github.com/smarterclayton)) 1263 - `PidPressure` now evicts pods from lowest priority to highest priority ([#72844](https://github.com/kubernetes/kubernetes/pull/72844), [@dashpole](https://github.com/dashpole)) 1264 - Split up the mondo `kubernetes-test` tarball into `kubernetes-test-portable` and `kubernetes-test-{OS}-{ARCH}` tarballs. ([#74065](https://github.com/kubernetes/kubernetes/pull/74065), [@ixdy](https://github.com/ixdy)) 1265 1266 ### VMware 1267 1268 - Applied zone labels to vSphere Volumes automatically. The zone labels are visible on the PV: ([#72687](https://github.com/kubernetes/kubernetes/pull/72687), [@subramanian-neelakantan](https://github.com/subramanian-neelakantan)) 1269 1270 ### Windows 1271 1272 Support for Windows nodes and Windows containers went going stable. 1273 1274 Support for Group Managed Service Accounts (GMSA) for Windows containers in Kubernetes. GMSA are a specific type of Active Directory account that provides automatic password management, simplified service principal name (SPN) management, and the ability to delegate the management to other administrators across multiple servers. 1275 1276 - Fixed smb remount and unmount issues on Windows ([#73661](https://github.com/kubernetes/kubernetes/pull/73661), [@andyzhangx](https://github.com/andyzhangx), [#75087](https://github.com/kubernetes/kubernetes/pull/75087), [@andyzhangx](https://github.com/andyzhangx)) 1277 - Added network stats for Windows nodes and containers ([#74788](https://github.com/kubernetes/kubernetes/pull/74788), [@feiskyer](https://github.com/feiskyer)) 1278 - The new test `[sig-network] DNS should now provide /etc/hosts entries for the cluster [LinuxOnly] [Conformance]` will validate the host entries set in the ``/etc/hosts`` file (pod's FQDN and hostname), which should be managed by Kubelet. ([#72729](https://github.com/kubernetes/kubernetes/pull/72729), [@bclau](https://github.com/bclau)) 1279 - Allowed the kubelet to pass Windows GMSA credentials down to Docker ([#73726](https://github.com/kubernetes/kubernetes/pull/73726), [@wk8](https://github.com/wk8)) 1280 - Added kube-proxy support for overlay networking and DSR in Windows and new flags for `network-name`, `source-vip`, and `enable-dsr`. ([#70896](https://github.com/kubernetes/kubernetes/pull/70896), [@ksubrmnn](https://github.com/ksubrmnn)) 1281 - windows: Ensured graceful termination when being run as windows service ([#73292](https://github.com/kubernetes/kubernetes/pull/73292), [@steffengy](https://github.com/steffengy)) 1282 - vSphere cloud provider now correctly retrieves the VM's UUID when running on Windows ([#71147](https://github.com/kubernetes/kubernetes/pull/71147), [@benmoss](https://github.com/benmoss)) 1283 - Kubelet: added `usageNanoCores` from CRI stats provider ([#73659](https://github.com/kubernetes/kubernetes/pull/73659), [@feiskyer](https://github.com/feiskyer)) 1284 - Introduced support for Windows nodes into the cluster bringup scripts for GCE. ([#73442](https://github.com/kubernetes/kubernetes/pull/73442), [@pjh](https://github.com/pjh)) 1285 - Added network stats for Windows nodes and pods. ([#70121](https://github.com/kubernetes/kubernetes/pull/70121), [@feiskyer](https://github.com/feiskyer)) 1286 - CoreDNS is only officially supported on Linux at this time. As such, when kubeadm is used to deploy this component into your kubernetes cluster, it will be restricted (using `nodeSelectors`) to run only on nodes with that operating system. This ensures that in clusters which include Windows nodes, the scheduler will not ever attempt to place CoreDNS pods on these machines, reducing setup latency and enhancing initial cluster stability. ([#69940](https://github.com/kubernetes/kubernetes/pull/69940), [@MarcPow](https://github.com/MarcPow)) 1287 1288 ## External Dependencies 1289 1290 - Default etcd server and client have been updated to v3.3.10. ([#71615](https://github.com/kubernetes/kubernetes/pull/71615), [#70168](https://github.com/kubernetes/kubernetes/pull/70168)) 1291 - The list of validated docker versions has changed. 1.11.1 and 1.12.1 have been removed. The current list is 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09. ([#72823](https://github.com/kubernetes/kubernetes/pull/72823), [#72831](https://github.com/kubernetes/kubernetes/pull/72831)) 1292 - The default Go version was updated to 1.12.1. ([#75422](https://github.com/kubernetes/kubernetes/pull/75422)) 1293 - CNI has been updated to v0.7.5 ([#75455](https://github.com/kubernetes/kubernetes/pull/75455)) 1294 - CSI has been updated to v1.1.0. ([#75391](https://github.com/kubernetes/kubernetes/pull/75391)) 1295 - The dashboard add-on has been updated to v1.10.1. ([#72495](https://github.com/kubernetes/kubernetes/pull/72495)) 1296 - Cluster Autoscaler has been updated to v1.14.0 ([#75480](https://github.com/kubernetes/kubernetes/pull/75480)) 1297 - kube-dns is unchanged at v1.14.13 since Kubernetes 1.12 ([#68900](https://github.com/kubernetes/kubernetes/pull/68900)) 1298 - Influxdb is unchanged at v1.3.3 since Kubernetes 1.10 ([#53319](https://github.com/kubernetes/kubernetes/pull/53319)) 1299 - Grafana is unchanged at v4.4.3 since Kubernetes 1.10 ([#53319](https://github.com/kubernetes/kubernetes/pull/53319)) 1300 - Kibana has been upgraded to v6.6.1. ([#71251](https://github.com/kubernetes/kubernetes/pull/71251)) 1301 - CAdvisor has been updated to v0.33.1 ([#75140](https://github.com/kubernetes/kubernetes/pull/75140)) 1302 - fluentd-gcp-scaler is unchanged at v0.5.0 since Kubernetes 1.13 ([#68837](https://github.com/kubernetes/kubernetes/pull/68837)) 1303 - Fluentd in fluentd-elasticsearch has been upgraded to v1.3.3 ([#71180](https://github.com/kubernetes/kubernetes/pull/71180)) 1304 - fluentd-elasticsearch has been updated to v2.4.0 ([#71180](https://github.com/kubernetes/kubernetes/pull/71180)) 1305 - The fluent-plugin-kubernetes_metadata_filter plugin in fluentd-elasticsearch has been updated to v2.1.6 ([#71180](https://github.com/kubernetes/kubernetes/pull/71180)) 1306 - fluentd-gcp is unchanged at v3.2.0 since Kubernetes 1.13 ([#70954](https://github.com/kubernetes/kubernetes/pull/70954)) 1307 - OIDC authentication is unchanged at coreos/go-oidc v2 since Kubernetes 1.10 ([#58544](https://github.com/kubernetes/kubernetes/pull/58544)) 1308 - Calico is unchanged at v3.3.1 since Kubernetes 1.13 ([#70932](https://github.com/kubernetes/kubernetes/pull/70932)) 1309 - crictl on GCE is unchanged at v1.12.0 since Kubernetes 1.13 ([#69033](https://github.com/kubernetes/kubernetes/pull/69033)) 1310 - CoreDNS has been updated to v1.3.1 ([#73610](https://github.com/kubernetes/kubernetes/pull/73610)) 1311 - event-exporter has been updated to v0.2.3 ([#67691](https://github.com/kubernetes/kubernetes/pull/67691)) 1312 - Es-image has been updated to Elasticsearch 6.6.1 ([#71252](https://github.com/kubernetes/kubernetes/pull/71252)) 1313 - metrics-server remains unchanged at v0.3.1 since Kubernetes 1.12 ([#68746](https://github.com/kubernetes/kubernetes/pull/68746)) 1314 - GLBC remains unchanged at v1.2.3 since Kubernetes 1.12 ([#66793](https://github.com/kubernetes/kubernetes/pull/66793)) 1315 - Ingress-gce remains unchanged at v1.2.3 since Kubernetes 1.12 ([#66793](https://github.com/kubernetes/kubernetes/pull/66793)) 1316 - ip-masq-agen remains unchanged at v2.1.1 since Kubernetes 1.12 ([#67916](https://github.com/kubernetes/kubernetes/pull/67916)) 1317 1318 - [v1.14.0-rc.1](#v1140-rc1) 1319 - [v1.14.0-beta.2](#v1140-beta2) 1320 - [v1.14.0-beta.1](#v1140-beta1) 1321 - [v1.14.0-alpha.3](#v1140-alpha3) 1322 - [v1.14.0-alpha.2](#v1140-alpha2) 1323 - [v1.14.0-alpha.1](#v1140-alpha1) 1324 1325 1326 1327 # v1.14.0-rc.1 1328 1329 [Documentation](https://docs.k8s.io) 1330 1331 ## Downloads for v1.14.0-rc.1 1332 1333 1334 filename | sha512 hash 1335 -------- | ----------- 1336 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes.tar.gz) | `5cb5e8b14b301864063fd7531ab3b755fea054f540c55ecce70ac49fb59193488575eb42ba89c8b4a44f6f2d005602ffc50ac286354a16df27637dd2e05f90f0` 1337 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-src.tar.gz) | `395424c2bcdb5e242995b18e8d6e5c00002ce2cb5a3964c28da0a4a181fada73ffceaccedb1fa9799be9b3c4fb5b451010cba65af4d7385c25c8c8f0298218fc` 1338 1339 ### Client Binaries 1340 1341 filename | sha512 hash 1342 -------- | ----------- 1343 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-darwin-386.tar.gz) | `ea304f3f8188db30cddd5423b25dc434b8f05315103f773619a65f83bee872581d83d5498a5f36a3064815e68998746cf661802eab36bfe96217253bac7e751c` 1344 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-darwin-amd64.tar.gz) | `17e106b63067429b9228a4879a7350c01ae98650ef2e6fcc23d00415c2e3a7c340abd5bcfc4b976f3d737d9268159ef5e5e7b08757371daad637e721a2ffb4a9` 1345 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-linux-386.tar.gz) | `406323ea4cbd524807e73b9a2f4eb0a813730b262402c224e5076080b84452137521e5782056e39bf6017bda8ef9e797ed497d51a653ed6822357f43d86bb0ae` 1346 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-linux-amd64.tar.gz) | `293747816e5da30c53ca29f27479fae880404edd5fca413af165e52cc7ea7ff26312bb3916896eceea75af6b232647268324da76d2d30fa2a4a688557427f7c7` 1347 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-linux-arm.tar.gz) | `98b6749c367282048ecd0c5b70ae8b7dafad82c599e359cbbe782a530bbd7bdb84a198577251d6aedbc39fb4e0297bd929d7e988eb557556f523073227375b09` 1348 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-linux-arm64.tar.gz) | `0a650c53946ebd9c38705df36efabbf1fbe3da1acdf418cc4ab881530e6a9089b45ef36bd4a89467106561e2165a00cc2c47f791d9ba422f36544bfa4b1e3b68` 1349 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-linux-ppc64le.tar.gz) | `807263f316f46c9334ec4dad84895cfe2b942ac4ed9aacf3ec8a63193e0d0a6ecaf573d00dfea278d1e552fe91e9474f534ea6798700dbdc84f1d9556ecfda3e` 1350 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-linux-s390x.tar.gz) | `5589562ce0da49f4987388e3e2b6fcf29e92859fae65fb57cdd61bba20ff574c7f3e07f4fa26bb63789f927690ce19710803e0d2e3324bd2c80ddc4925ea973d` 1351 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-windows-386.tar.gz) | `101245cd70221b443150be046e5b5a2c6c83334085bb17f603f59bb68c5960f353d57b6761acf052449f4fb057a5525978cdd7995d06134ab98f8628c23aac0d` 1352 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-client-windows-amd64.tar.gz) | `b1c3802f60cc70ebc1258cb8fc4ffa1154ecee8fda473b033be4f9d1b187354cfd75d085a1ad45a35fcc42ad640c4fecaa2496cb48fad649a774f40a5150825f` 1353 1354 ### Server Binaries 1355 1356 filename | sha512 hash 1357 -------- | ----------- 1358 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-server-linux-amd64.tar.gz) | `8e09465aab0a1d1ec39afc98af17de9b5de99b763c0c5feb2dc824f2bbae25edb690e9d162fd44f5155bea392bd229f544850fe19e767a8b342050f4bfafc2cc` 1359 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-server-linux-arm.tar.gz) | `9409c368e1f9f26e633b7df5d6c90435394931d48a21f4ad1548d172b18ac462a859019cc66dba4df69d3b10702820c9a3e1bd5c469646b1db581e52e79e035d` 1360 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-server-linux-arm64.tar.gz) | `3893290dc11ca71746fee77a44a607ad9e02036bab56b7fc3be247b71b2cf5b3f639fa41317a713abbe9a997abf80c7113ff1155482d0cc04a318ec8beccf869` 1361 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-server-linux-ppc64le.tar.gz) | `8ac1e70cb39aeb5b1fe92c3ebba428db2036be739f462cc2f876f17dc71a01ba263b5611a15d95e5934e2fc7aa92042bc9b2fe65cf459263be90f5fbb5d83a15` 1362 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-server-linux-s390x.tar.gz) | `80576cb6cc3a69c4fc0a0358dee5772ecb38437c534a3454c9613426417d4af3c527a0809cce4d46653a7b001c58033b06326c80c498d17387569d22d3ca9b22` 1363 1364 ### Node Binaries 1365 1366 filename | sha512 hash 1367 -------- | ----------- 1368 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-node-linux-amd64.tar.gz) | `71cd5dd6075a2eea851231a5a855e58b3f479d83358defafd068dd1d09e5b2c426a8a046ee621de91e17d7ecd67465911b93549088bf27a41c6e6b77d692a8c7` 1369 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-node-linux-arm.tar.gz) | `48babf4a52013c2bc69049167579ba1bc70c769b782a2704c9dfcf44a6a8a72f07e0789af347135ee4797f2bf1a216c348a9a4a26be71855e95e8387bf4e2aac` 1370 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-node-linux-arm64.tar.gz) | `cc19a52beaa7440ca7581e85d1e10137e93c2decdb7d7d7919e7fdcded63e4d94b3434513ca881dae844dd1eb1e2fe98ea5332fad5ffced846f729894ecf0ed1` 1371 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-node-linux-ppc64le.tar.gz) | `241bf20ae366384efa0fb3798e07e1cdd3d4ea7ba91c146ef7761fb0b93a8514a0dd91f9eb47999ae263d6793e0577c2bbecf88548bcae06cedb437331d6d3bc` 1372 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-node-linux-s390x.tar.gz) | `37ce3e021073c6c10cf240fb2c3f9a7ab35ece3c0b4a9fecbbdf790eb348b168d179824f3a8eb57d56f962b64f8a6a71925152c087f5bfe43b004cbfae65674d` 1373 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-rc.1/kubernetes-node-windows-amd64.tar.gz) | `3248db12c274171f094ef41d6a59523aec35655ba35d151576ff2d2089c269f345e2c0646d585d7c04c440c6b4e7379c499bc8f51fcc8b91388e3dc4d1b6a899` 1374 1375 ## Changelog since v1.14.0-beta.2 1376 1377 ### Action Required 1378 1379 * ACTION REQUIRED: Health check (liveness & readiness) probes using an HTTPGetAction will no longer follow redirects to different hostnames from the original probe request. Instead, these non-local redirects will be treated as a Success (the documented behavior). In this case an event with reason "ProbeWarning" will be generated, indicating that the redirect was ignored. If you were previously relying on the redirect to run health checks against different endpoints, you will need to perform the healthcheck logic outside the Kubelet, for instance by proxying the external endpoint rather than redirecting to it. ([#75416](https://github.com/kubernetes/kubernetes/pull/75416), [@tallclair](https://github.com/tallclair)) 1380 1381 ### Other notable changes 1382 1383 * Restores --username and --password flags to kubectl ([#75451](https://github.com/kubernetes/kubernetes/pull/75451), [@liggitt](https://github.com/liggitt)) 1384 * fix race condition issue for smb mount on windows ([#75371](https://github.com/kubernetes/kubernetes/pull/75371), [@andyzhangx](https://github.com/andyzhangx)) 1385 * UDP Service conntrack entries for ExternalIPs are now correctly cleared when endpoints are added ([#75265](https://github.com/kubernetes/kubernetes/pull/75265), [@JacobTanenbaum](https://github.com/JacobTanenbaum)) 1386 * kubeadm: the kubeadm init output now provides join control-plane example only when the preconditions for joining a control plane are satisfied ([#75420](https://github.com/kubernetes/kubernetes/pull/75420), [@fabriziopandini](https://github.com/fabriziopandini)) 1387 * Fix dockershim panic issues when deleting docker images. ([#75367](https://github.com/kubernetes/kubernetes/pull/75367), [@feiskyer](https://github.com/feiskyer)) 1388 * kubeadm: Allow certain certs/keys to be missing on the secret when transferring secrets using `--experimental-upload-certs` feature ([#75415](https://github.com/kubernetes/kubernetes/pull/75415), [@ereslibre](https://github.com/ereslibre)) 1389 * Update to use go1.12.1 ([#75413](https://github.com/kubernetes/kubernetes/pull/75413), [@BenTheElder](https://github.com/BenTheElder)) 1390 * Update CSI version to 1.1 ([#75391](https://github.com/kubernetes/kubernetes/pull/75391), [@gnufied](https://github.com/gnufied)) 1391 * Ensure ownership when deleting a load balancer security group ([#74311](https://github.com/kubernetes/kubernetes/pull/74311), [@hpedrorodrigues](https://github.com/hpedrorodrigues)) 1392 * kubelet: updated logic of verifying a static critical pod. ([#75144](https://github.com/kubernetes/kubernetes/pull/75144), [@Huang-Wei](https://github.com/Huang-Wei)) 1393 * Allow disable outbound SNAT when Azure standard load balancer is used together with outbound rules. ([#75282](https://github.com/kubernetes/kubernetes/pull/75282), [@feiskyer](https://github.com/feiskyer)) 1394 * Add ResourceVersion as a precondition for delete in order to ensure a delete fails if an unobserved change happens to an object. ([#74040](https://github.com/kubernetes/kubernetes/pull/74040), [@ajatprabha](https://github.com/ajatprabha)) 1395 * Services of type=LoadBalancer which have no endpoints will now immediately ICMP reject connections, rather than time out. ([#74394](https://github.com/kubernetes/kubernetes/pull/74394), [@thockin](https://github.com/thockin)) 1396 * Ensure Azure load balancer cleaned up on 404 or 403 when deleting LoadBalancer services. ([#75256](https://github.com/kubernetes/kubernetes/pull/75256), [@feiskyer](https://github.com/feiskyer)) 1397 1398 1399 1400 # v1.14.0-beta.2 1401 1402 [Documentation](https://docs.k8s.io) 1403 1404 ## Downloads for v1.14.0-beta.2 1405 1406 1407 filename | sha512 hash 1408 -------- | ----------- 1409 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes.tar.gz) | `c1d5f2615c3319fc167c577f40f385abe6652bf4fd3bdb04617b36029dc3000b190c18b4b3a29827da75c680979697d61fffb45b86ba6226f880b98b2f308f4f` 1410 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-src.tar.gz) | `0a8d8ed208bc0bf424060126c76fcd8dbbd53a9b9695647314a4097f7013f548b76850438933760ff76835867676cddddf65134ad79f977ecdb98632fc2edda3` 1411 1412 ### Client Binaries 1413 1414 filename | sha512 hash 1415 -------- | ----------- 1416 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-darwin-386.tar.gz) | `c919d030255c5d3879926d8aaa53939cd5aa37084799748452166ca6668bd1d10edf063d633682cddafaaed43dd1b991f4ad09139c5e4f519bd69f581b3fe0aa` 1417 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-darwin-amd64.tar.gz) | `ec14d4a1d720890065211544b099be17315265534cfd20435194dc842cc807c20b5fae78f5b95ba7d05f3d921d522017f50861760d195ce1bf5b1acfdb2dbb29` 1418 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-linux-386.tar.gz) | `6cee12be5b855600ee80f15d1e0511099941b099bd5b252549abdc2a65c077f10ca4d53ab9804a0ce8d51f3b9cbab829cb551733cd4aed37c0d91238b82a8fe4` 1419 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-linux-amd64.tar.gz) | `27d8cd48c1f3259055965b85a6b973ecb5b8a36894f94c232d773f89539e28e6c270bfe35427c70b4ad4800e42c869851981cf88f586b7d488efa538e6c88126` 1420 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-linux-arm.tar.gz) | `7f98230569c61fdf2b141e519f042b2e27ff37aeda746dc30bb7ce226b5d6b0c0bb85c6070b9ffc8d38c2441feb5bd8736c67708a59552e86a2c30cc02ecdece` 1421 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-linux-arm64.tar.gz) | `159da67010af38d87c5318b7ad594120afd6a9b780d11d6e607e7214862cd6514b00da673cce72574771dbc780ab435dbba0a3267f051a20155c05ee0729ded0` 1422 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-linux-ppc64le.tar.gz) | `d8ca7871d3d40947db69061284cb31c4d072d4da56fbb11a4485f6853f041835d9605cfc5dcea88d58c7f484cce13dbca485e80891c845291b9b28c574df310b` 1423 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-linux-s390x.tar.gz) | `1c58db90b6e09b8d8f956a00263cb20271b8403f7fb6c5b20d76cca9ed973c35d2f5c910a6d42980ec9159480682d3786a59e9e05ce356a7e3b4181c848ad122` 1424 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-windows-386.tar.gz) | `00fb87dd4899208dd6607c22828f3985ebfd5e1f97cb24e3b2c69c249a4887d5c26c603b3bb4c21f9e2b737c917ddf95a1818d9de5c9ec97d3f5faa0c3dada52` 1425 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-client-windows-amd64.tar.gz) | `7afdf637d62dde480162ad1521360b2bc78e0d4d20f6e6201e2f19b55b8e9bbd69c1ce8d03101c750ef389c65a1bc0a94dfc9a2d501d6840fd31eacbd3582028` 1426 1427 ### Server Binaries 1428 1429 filename | sha512 hash 1430 -------- | ----------- 1431 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-server-linux-amd64.tar.gz) | `2ac3c4910cd36f02a62304d78fe144b821edf445c522028e6b57d2dc3bcc7355159a58815d5a6991b3f2c33bb0ef23e07134c8bbf93b34be7452f80c9a9e6edf` 1432 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-server-linux-arm.tar.gz) | `db06b5f1a83ca4ec82428ab771eae2858b188dc23780fb9b146494c06aa6175421090b200c58b670e2d4253a7e0d4b07172a632e0754c35ccdee7e264c636f17` 1433 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-server-linux-arm64.tar.gz) | `b40e1745d1ecfcc95f3a750990244fa128381d6d74246798a62aecb8cec9c77cdcd470e79334eb5c670e1e3a288080b4e26a080c64481ba608e3156c72df474b` 1434 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-server-linux-ppc64le.tar.gz) | `c84297fd2b18b6bdade5a135a3da929e286bbba5c8dd66778091bad4eea1ac4b97a32ab3b146a88f0716bebcce9a4a85a7cd421cb185a3df864dcfa77312b3a9` 1435 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-server-linux-s390x.tar.gz) | `39c8f6e7f52bec155b11652b4e80c2c52acf8754dbdf80a9d5bab5370d1debf4f4783c1a6968d41822b00ff744c72947df6cbc4623578e7679b9ce9a98f64ff3` 1436 1437 ### Node Binaries 1438 1439 filename | sha512 hash 1440 -------- | ----------- 1441 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-node-linux-amd64.tar.gz) | `bf19ae7140836aea1b6f414532eff886e3b91e0746b9224ce46e60e0b83fa90a8c3df1ff8e01ff340a1e1874ce15da28e98224024ed3139589474e89befa19f2` 1442 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-node-linux-arm.tar.gz) | `4b8194340a8675107da3969845173fb34ed2b0a38745ec0ac395ebf2116ee84d55be6e22ff84fafbaf4ca60a05f6debf6e95957a2261ac8a587eea32e5803fd5` 1443 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-node-linux-arm64.tar.gz) | `8494dae5225f3b543afd575003fe0f30eb0f3cf9bc9dfbae72d6bda8f17c5446165433c28842a114af66ac3ae8fca9f92d780d1eb93e9bcb6b5dcc4fe8cd2a7f` 1444 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-node-linux-ppc64le.tar.gz) | `0bd41d95f0a76c1b057a8913a8b2cffbf6d48c47aef1d9beed0de205b8010e8071e8f527eeaa003730ef97a017083278cb2036cf22a1abfb2f4669b935823cdc` 1445 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-node-linux-s390x.tar.gz) | `169b16512df42a6cb5b000a3d6d6da5ae48a733c5d11b034eaec6b3816b86ec97b92e4075872900188bc296427037299841224e552ccf079097d5cf333627cbc` 1446 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.2/kubernetes-node-windows-amd64.tar.gz) | `ccfebde5176cb28529552889250bf706add04df3c3f9aee5b8bdd9ebdb5bce334add8351cdbeebb9bd4b17b31d994b7739d2c494aa4c64bcb3ceba1a6ef53447` 1447 1448 ## Changelog since v1.14.0-beta.1 1449 1450 ### Action Required 1451 1452 * ACTION REQUIRED: The node.k8s.io API group and runtimeclasses.node.k8s.io resource have been migrated to a built-in API. If you were using RuntimeClasses (an default-disabled alpha feature, as of Kubernetes v1.12), then you must recreate all RuntimeClasses after upgrading, and the runtimeclasses.node.k8s.io CRD should be manually deleted. RuntimeClasses can no longer be created without a defined handler. ([#74433](https://github.com/kubernetes/kubernetes/pull/74433), [@tallclair](https://github.com/tallclair)) 1453 * Introduce a RuntimeClass v1beta1 API. This new beta API renames `runtimeHandler` to `handler`, makes it a required field, and cuts out the spec (handler is a top-level field). 1454 * Transition CSINodeInfo and CSIDriver alpha CRDs to in-tree CSINode and CSIDriver core storage v1beta1 APIs. ([#74283](https://github.com/kubernetes/kubernetes/pull/74283), [@xing-yang](https://github.com/xing-yang)) 1455 * ACTION REQUIRED: the alpha CRDs are no longer used and drivers will need to be updated to use the beta APIs. 1456 * The support for `_` in the CSI driver name will be dropped as the CSI Spec does not allow that. 1457 1458 ### Other notable changes 1459 1460 * Support collecting pod logs under /var/log/pods/NAMESPACE_NAME_UID to stackdriver with `k8s_pod` resource type. ([#74502](https://github.com/kubernetes/kubernetes/pull/74502), [@Random-Liu](https://github.com/Random-Liu)) 1461 * --make-symlinks for hyperkube was marked hidden for a while, This flag is now deprecated and will be removed in a future release. ([#74975](https://github.com/kubernetes/kubernetes/pull/74975), [@dims](https://github.com/dims)) 1462 * fix smb unmount issue on Windows ([#75087](https://github.com/kubernetes/kubernetes/pull/75087), [@andyzhangx](https://github.com/andyzhangx)) 1463 * Kubelet no longer watches configmaps and secrets for terminated pods, in worst scenario causing it to not be able to send other requests to kube-apiserver ([#74809](https://github.com/kubernetes/kubernetes/pull/74809), [@oxddr](https://github.com/oxddr)) 1464 * Fixes a bug concerning Quobyte volumes where user mappings only worked if the hosts Kubernetes plugin mount was provided via an external configuration using the _allow-usermapping-in-volumename_ option. ([#74520](https://github.com/kubernetes/kubernetes/pull/74520), [@casusbelli](https://github.com/casusbelli)) 1465 * Change CRI pod log directory from `/var/log/pods/UID` to `/var/log/pods/NAMESPACE_NAME_UID`. ([#74441](https://github.com/kubernetes/kubernetes/pull/74441), [@Random-Liu](https://github.com/Random-Liu)) 1466 * It is recommended to drain the node before upgrade, or reboot the node after upgrade. 1467 * Promote RuntimeClass to beta, and enable by default. ([#75003](https://github.com/kubernetes/kubernetes/pull/75003), [@tallclair](https://github.com/tallclair)) 1468 * New "dry_run" metric label (indicating the value of the dryRun query parameter) into the metrics: ([#74997](https://github.com/kubernetes/kubernetes/pull/74997), [@jennybuckley](https://github.com/jennybuckley)) 1469 * apiserver_request_total 1470 * apiserver_request_duration_seconds 1471 * New "APPLY" value for the "verb" metric label which indicates a PATCH with "Content-Type: apply-patch+yaml". This value is experimental and will only be present if the ServerSideApply alpha feature is enabled. 1472 * GCE: bump COS image version to cos-beta-73-11647-64-0 ([#75149](https://github.com/kubernetes/kubernetes/pull/75149), [@yguo0905](https://github.com/yguo0905)) 1473 * Add duration metric for CRD webhook converters ([#74376](https://github.com/kubernetes/kubernetes/pull/74376), [@mbohlool](https://github.com/mbohlool)) 1474 * Alpha support for ephemeral CSI inline volumes that are embedded in pod specs. ([#74086](https://github.com/kubernetes/kubernetes/pull/74086), [@vladimirvivien](https://github.com/vladimirvivien)) 1475 * Add support for node side CSI volume expansion ([#74863](https://github.com/kubernetes/kubernetes/pull/74863), [@gnufied](https://github.com/gnufied)) 1476 * Add mechanism for Admission Webhooks to specify which version of AdmissionReview they support ([#74998](https://github.com/kubernetes/kubernetes/pull/74998), [@mbohlool](https://github.com/mbohlool)) 1477 * Add mechanism for CRD Conversion Webhooks to specify which version of ConversionReview they support 1478 * Add a new kubelet endpoint for serving first-class resource metrics ([#73946](https://github.com/kubernetes/kubernetes/pull/73946), [@dashpole](https://github.com/dashpole)) 1479 * Deprecate AWS, Azure, GCE and Cinder specific volume limit predicates. ([#74544](https://github.com/kubernetes/kubernetes/pull/74544), [@gnufied](https://github.com/gnufied)) 1480 * PodReadinessGate feature is now GA. The feature gate will not allow disabling it. ([#74434](https://github.com/kubernetes/kubernetes/pull/74434), [@freehan](https://github.com/freehan)) 1481 * If CSINodeInfo and CSIMigration feature flags are active in the cluster, Kubelet will post NotReady until CSINode is initialized with basic volume plugin mechanism information for well-known drivers ([#74835](https://github.com/kubernetes/kubernetes/pull/74835), [@davidz627](https://github.com/davidz627)) 1482 * Add network stats for Windows nodes and containers ([#74788](https://github.com/kubernetes/kubernetes/pull/74788), [@feiskyer](https://github.com/feiskyer)) 1483 * kubeadm: when calling "reset" on a control-plane node, remove the APIEndpoint information for this node from the ClusterStatus in the kubeadm ConfigMap. ([#75082](https://github.com/kubernetes/kubernetes/pull/75082), [@neolit123](https://github.com/neolit123)) 1484 * kube-apiserver now serves OpenAPI specs for registered CRDs with defined ([#71192](https://github.com/kubernetes/kubernetes/pull/71192), [@roycaihw](https://github.com/roycaihw)) 1485 * validation schemata as an alpha feature, to be enabled via the "CustomResourcePublishOpenAPI" feature gate. Kubectl will validate client-side using those. Note that in 1486 * future, client-side validation in 1.14 kubectl against a 1.15 cluster will reject 1487 * unknown fields for CRDs with validation schema defined. 1488 * Fix kubelet start failure issue on Azure Stack due to InstanceMetadata setting ([#74936](https://github.com/kubernetes/kubernetes/pull/74936), [@rjaini](https://github.com/rjaini)) 1489 * add subcommand `kubectl create cronjob` ([#71651](https://github.com/kubernetes/kubernetes/pull/71651), [@Pingan2017](https://github.com/Pingan2017)) 1490 * The CSIBlockVolume feature gate is now beta, and defaults to enabled. ([#74909](https://github.com/kubernetes/kubernetes/pull/74909), [@bswartz](https://github.com/bswartz)) 1491 * Pre-existing log files are now opened with O_APPEND, instead of O_TRUNC. This helps prevent losing logs when components crash-loop, and also enables external log rotation utilities to truncate log files in-place without components extending log files to their pre-truncation sizes on subsequent writes. ([#74837](https://github.com/kubernetes/kubernetes/pull/74837), [@mtaufen](https://github.com/mtaufen)) 1492 * the test/e2e/e2e.test binary can test arbitrary storage drivers, see the `-storage.testdriver` parameter ([#72836](https://github.com/kubernetes/kubernetes/pull/72836), [@pohly](https://github.com/pohly)) 1493 * Fix panic in kubectl cp command ([#75037](https://github.com/kubernetes/kubernetes/pull/75037), [@soltysh](https://github.com/soltysh)) 1494 * iscsi modules haven't even been loaded /sys/class/iscsi_host directory won't exist ([#74787](https://github.com/kubernetes/kubernetes/pull/74787), [@jianglingxia](https://github.com/jianglingxia)) 1495 * the fluentd addon daemonset will now target all nodes. ([#74424](https://github.com/kubernetes/kubernetes/pull/74424), [@liggitt](https://github.com/liggitt)) 1496 * setting `ENABLE_METADATA_CONCEALMENT=true` in kube-up will now set a `cloud.google.com/metadata-proxy-ready=true` label on new nodes. In v1.16, the metadata proxy add-on will switch to using that label as a node selector. 1497 * setting `KUBE_PROXY_DAEMONSET=true` in kube-up will now set a `node.kubernetes.io/kube-proxy-ds-ready=true` label on new nodes. In v1.16, the kube-proxy daemonset add-on will switch to using that label as a node selector. 1498 * In 1.16, the masq-agent daemonset add-on will switch to using `node.kubernetes.io/masq-agent-ds-ready` as a node selector. 1499 * Kubelet: replace `du` and `find` with a golang implementation ([#74675](https://github.com/kubernetes/kubernetes/pull/74675), [@dashpole](https://github.com/dashpole)) 1500 * Kubelet: periodically update machine info to support hot-add/remove 1501 * kubeadm: add certificate-key and skip-certificate-key-print flags to kubeadm init ([#74671](https://github.com/kubernetes/kubernetes/pull/74671), [@yagonobre](https://github.com/yagonobre)) 1502 * Admission webhooks rules can now limit scope to only match namespaced, or only cluster-scoped resources with a `scope: "Cluster" | "Namespaced" | "*"` field. ([#74477](https://github.com/kubernetes/kubernetes/pull/74477), [@liggitt](https://github.com/liggitt)) 1503 * The CSIPersistentVolume and KubeletPluginWatcher feature gates cannot be disabled, and will be removed in Kubernetes v1.16 ([#74830](https://github.com/kubernetes/kubernetes/pull/74830), [@msau42](https://github.com/msau42)) 1504 * Kubelet won't evict a static pod with priority `system-node-critical` upon resource pressure. ([#74222](https://github.com/kubernetes/kubernetes/pull/74222), [@Huang-Wei](https://github.com/Huang-Wei)) 1505 * Fixes panic if a kubelet is run against an older kube-apiserver ([#74529](https://github.com/kubernetes/kubernetes/pull/74529), [@liggitt](https://github.com/liggitt)) 1506 * The resource group name in Azure providerID is not converted to lower cases. ([#74882](https://github.com/kubernetes/kubernetes/pull/74882), [@feiskyer](https://github.com/feiskyer)) 1507 * Remove the out-of-tree PersistentVolumeLabel controller because it cannot run without Initializers (removed in v1.14). If you are using AWS EBS, GCE PD, Azure Disk, Cinder Disk or vSphere volumes and rely on zone labels, then enable the `PersistentVolumeLabel` admission controller in the `kube-apiserver` in the `--enable-admission-plugins` flag. ([#74615](https://github.com/kubernetes/kubernetes/pull/74615), [@andrewsykim](https://github.com/andrewsykim)) 1508 * kubeadm: improved RequiredIPVSKernelModulesAvailable warning message ([#74033](https://github.com/kubernetes/kubernetes/pull/74033), [@bart0sh](https://github.com/bart0sh)) 1509 * Add `nullable` support to CustomResourceDefinition OpenAPI validation schemata. ([#74804](https://github.com/kubernetes/kubernetes/pull/74804), [@sttts](https://github.com/sttts)) 1510 * Fix kube-apiserver not to create default/kubernetes service endpoints before it reports readiness via the /healthz and therefore is ready to serve requests. Also early during startup old endpoints are remove which might be left over from a previously crashed kube-apiserver. ([#74668](https://github.com/kubernetes/kubernetes/pull/74668), [@sttts](https://github.com/sttts)) 1511 * kubeadm: fix a bug where standard kubeconfig paths were searched even if the user provided /etc/kubernetes/admin.conf explicitly for commands that accept --kubeconfig, like kubeadm token. ([#71874](https://github.com/kubernetes/kubernetes/pull/71874), [@neolit123](https://github.com/neolit123)) 1512 * kubeadm: use the default kubeconfig (/etc/kubernetes/admin.conf) for "kubeadm reset" and "kubeadm upgrade" commands. 1513 * Increase api server client certificate expiration histogram resolution to accommodate short-lived (< 6h) client certificates. ([#74806](https://github.com/kubernetes/kubernetes/pull/74806), [@mxinden](https://github.com/mxinden)) 1514 * Default RBAC policy no longer grants access to discovery and permission-checking APIs (used by `kubectl auth can-i`) to *unauthenticated* users. Upgraded clusters preserve prior behavior, but cluster administrators wishing to grant unauthenticated users access in new clusters will need to explicitly opt-in to expose the discovery and/or permission-checking APIs: ([#73807](https://github.com/kubernetes/kubernetes/pull/73807), [@dekkagaijin](https://github.com/dekkagaijin)) 1515 * `kubectl create clusterrolebinding anonymous-discovery --clusterrole=system:discovery --group=system:unauthenticated` 1516 * `kubectl create clusterrolebinding anonymous-access-review --clusterrole=system:basic-user --group=system:unauthenticated` 1517 * The PersistentLocalVolumes feature is GA. The feature gate cannot be disabled and will be removed in Kubernetes 1.17 ([#74769](https://github.com/kubernetes/kubernetes/pull/74769), [@msau42](https://github.com/msau42)) 1518 * kubelet: resolved hang/timeout issues when running large numbers of pods with unique configmap/secret references by reverting to 1.11 configmap/secret lookup behavior ([#74755](https://github.com/kubernetes/kubernetes/pull/74755), [@liggitt](https://github.com/liggitt)) 1519 * Convert `latency`/`latencies` in metrics name to `duration`. ([#74418](https://github.com/kubernetes/kubernetes/pull/74418), [@danielqsj](https://github.com/danielqsj)) 1520 * The following metrics are changed and mark previous metrics as deprecated: 1521 * `rest_client_request_latency_seconds` -> `rest_client_request_duration_seconds` 1522 * `apiserver_proxy_tunnel_sync_latency_secs` -> `apiserver_proxy_tunnel_sync_duration_seconds` 1523 * `scheduler_scheduling_latency_seconds` -> `scheduler_scheduling_duration_seconds ` 1524 * Fix help message for --container-runtime-endpoint: only unix socket is support on Linux. ([#74712](https://github.com/kubernetes/kubernetes/pull/74712), [@feiskyer](https://github.com/feiskyer)) 1525 * Update to use golang 1.12 ([#74632](https://github.com/kubernetes/kubernetes/pull/74632), [@cblecker](https://github.com/cblecker)) 1526 * The `RunAsGroup` feature has been promoted to beta and enabled by default. PodSpec and PodSecurityPolicy objects can be used to control the primary GID of containers on supported container runtimes. ([#73007](https://github.com/kubernetes/kubernetes/pull/73007), [@krmayankk](https://github.com/krmayankk)) 1527 * fix Azure Container Registry anonymous repo image pull error ([#74715](https://github.com/kubernetes/kubernetes/pull/74715), [@andyzhangx](https://github.com/andyzhangx)) 1528 * Adds the same information to an init container as a standard container in a pod when using PodPresets. ([#71479](https://github.com/kubernetes/kubernetes/pull/71479), [@soggiest](https://github.com/soggiest)) 1529 * fix the flake in scheduling_queue_test.go ([#74611](https://github.com/kubernetes/kubernetes/pull/74611), [@denkensk](https://github.com/denkensk)) 1530 * The kube-apiserver OpenAPI definitions with the prefix "io.k8s.kubernetes.pkg" (deprecated since 1.9) have been removed. ([#74596](https://github.com/kubernetes/kubernetes/pull/74596), [@sttts](https://github.com/sttts)) 1531 * kube-conformance image will now run ginkgo with the --dryRun flag if the container is run with the environment variable E2E_DRYRUN set. ([#74731](https://github.com/kubernetes/kubernetes/pull/74731), [@johnSchnake](https://github.com/johnSchnake)) 1532 * The deprecated `MountPropagation` feature gate has been removed, and the feature is now unconditionally enabled. ([#74720](https://github.com/kubernetes/kubernetes/pull/74720), [@bertinatto](https://github.com/bertinatto)) 1533 * Introduce dynamic volume provisioning shim for CSI migration ([#73653](https://github.com/kubernetes/kubernetes/pull/73653), [@ddebroy](https://github.com/ddebroy)) 1534 * Fix --help flag parsing ([#74682](https://github.com/kubernetes/kubernetes/pull/74682), [@soltysh](https://github.com/soltysh)) 1535 * This PR removes the following metrics: ([#74636](https://github.com/kubernetes/kubernetes/pull/74636), [@logicalhan](https://github.com/logicalhan)) 1536 * reflector_items_per_list 1537 * reflector_items_per_watch 1538 * reflector_last_resource_version 1539 * reflector_list_duration_seconds 1540 * reflector_lists_total 1541 * reflector_short_watches_total 1542 * reflector_watch_duration_seconds 1543 * reflector_watches_total 1544 * While this is a backwards-incompatible change, it would have been impossible to setup reliable monitoring around these metrics since the labels were not stable. 1545 * Add a configuration field to shorten the timeout of validating/mutating admission webhook call. The timeout value must be between 1 and 30 seconds. Default to 30 seconds when unspecified. ([#74562](https://github.com/kubernetes/kubernetes/pull/74562), [@roycaihw](https://github.com/roycaihw)) 1546 * client-go: PortForwarder.GetPorts() now contain correct local port if no local port was initially specified when setting up the port forwarder ([#73676](https://github.com/kubernetes/kubernetes/pull/73676), [@martin-helmich](https://github.com/martin-helmich)) 1547 * The examples in kubectl apply/get/delete are updated to support `-k` which uses a `kustomization.yaml` file. ([#74140](https://github.com/kubernetes/kubernetes/pull/74140), [@Liujingfang1](https://github.com/Liujingfang1)) 1548 * kubeadm: Allow to download certificate secrets uploaded by `init` or `upload-certs` phase, allowing to transfer certificate secrets (certificates and keys) from the cluster to other master machines when creating HA deployments. ([#74168](https://github.com/kubernetes/kubernetes/pull/74168), [@ereslibre](https://github.com/ereslibre)) 1549 * Fixes an issue with missing apiVersion/kind in object data sent to admission webhooks ([#74448](https://github.com/kubernetes/kubernetes/pull/74448), [@liggitt](https://github.com/liggitt)) 1550 * client-go: the deprecated versionless API group accessors (like `clientset.Apps()` have been removed). Use an explicit version instead (like `clientset.AppsV1()`) ([#74422](https://github.com/kubernetes/kubernetes/pull/74422), [@liggitt](https://github.com/liggitt)) 1551 * The `--quiet` option to `kubectl run` now suppresses resource deletion messages emitted when the `--rm` option is specified. ([#73266](https://github.com/kubernetes/kubernetes/pull/73266), [@awh](https://github.com/awh)) 1552 * Add Custom Resource support to "kubectl autoscale" ([#72678](https://github.com/kubernetes/kubernetes/pull/72678), [@rmohr](https://github.com/rmohr)) 1553 * Image garbage collection no longer fails for images with only one tag but more than one repository associated. ([#70647](https://github.com/kubernetes/kubernetes/pull/70647), [@corvus-ch](https://github.com/corvus-ch)) 1554 * Fix liveness probe in fluentd-gcp cluster addon ([#74522](https://github.com/kubernetes/kubernetes/pull/74522), [@Pluies](https://github.com/Pluies)) 1555 * The new test ``[sig-network] DNS should provide /etc/hosts entries for the cluster [LinuxOnly] [Conformance]`` will validate the host entries set in the ``/etc/hosts`` file (pod's FQDN and hostname), which should be managed by Kubelet. ([#72729](https://github.com/kubernetes/kubernetes/pull/72729), [@bclau](https://github.com/bclau)) 1556 * The test has the tag ``[LinuxOnly]`` because individual files cannot be mounted in Windows Containers, which means that it cannot pass using Windows nodes. 1557 1558 1559 1560 # v1.14.0-beta.1 1561 1562 [Documentation](https://docs.k8s.io) 1563 1564 ## Downloads for v1.14.0-beta.1 1565 1566 1567 filename | sha512 hash 1568 -------- | ----------- 1569 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes.tar.gz) | `065014c751635f8c077fbcc105df578594baf8afd8b8339004909198e1bd68d0a7ce3644ed5d54e5964d1306aa650f22a5ce83063415240f4dffd6706c1cc33b` 1570 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-src.tar.gz) | `244c19d9fad21ae154ee78fc94888dc60bcfcf3ec72bdc28a82e77c572cbc969d2abbf20397ef9564a35585c08dfe179b105fc25efac973e0a13d78ee2ff8f42` 1571 1572 ### Client Binaries 1573 1574 filename | sha512 hash 1575 -------- | ----------- 1576 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-darwin-386.tar.gz) | `78a61a1e922daa39a9f7dd61b8bad87d202b537bda59f90ae8aae941c0ff412e3d328530af9dd9f22462cbd67254e7ce1556defe48bb10bd6a94d4302464fe8e` 1577 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-darwin-amd64.tar.gz) | `5a1d66fd90a5dc07b95b7617b5583595e0a4a664e7005f6281f846c85b21e28692b2e2d55b7c40c7b8cbb96b6b8cd6c832f340c7cc67579641beabac033014ad` 1578 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-linux-386.tar.gz) | `30991ce0776bae7551b98a811e3ccb5104b0859805c41a216db7d5779cddfb36ef3c5e658ea2adeaf67f8e6f181768850b09a0e8320c2983d34664156ea638ac` 1579 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-linux-amd64.tar.gz) | `5501a9c3a95e43f0c691b05043546f2c497d50a6ad88b88219842d61be83d7dcb8871ff9fd2447c02bc842c4c962f298a3ce2e0618fae70e8aa391c9407626e5` 1580 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-linux-arm.tar.gz) | `4bf341a2f943719d006f4cacc26fdf4d021560d37d49c8d9c4620d294142041155a88dc721d9373a8617e1baa904c02b4545f379ffe87c6ab20e5459a5d3c2de` 1581 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-linux-arm64.tar.gz) | `1c0c660618947b053404ab8ef40bdec0a06d54a1f9edc585a7259806f878327d9ae54100bba98e5b7f44f5db4303276d189792e68517603520a49868c07e684c` 1582 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-linux-ppc64le.tar.gz) | `9d2c3f4bfbad03b41859670f9cfda6596f51a89077fd4da2f74490f71b5de10e459954e897d2a1ba3a217c62caaf1be74424e5bf6a5609868dc4f069ac06c94f` 1583 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-linux-s390x.tar.gz) | `9be5259caf39ff3c4d0f024d616bff50ac417d42a87c56b6877fbdf5aaadecee05631ac8ddd6dca20d52ae4ca7e1227a1fd5e882be9821f3711e144b84d41099` 1584 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-windows-386.tar.gz) | `5b1e75f532d9a4d656cd1c5ec48a19d01c4ca731c6c3d986ebbc48a9f1e1d61d6c5603145808fb929117cf2202bc75f31ec7661a50edf24c5af6b3419ebcc0ab` 1585 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-client-windows-amd64.tar.gz) | `a501d3c0be55d5a73214a83d1f48b39a49d4a84f5f9988fa34cf66ef6bc78d3f3e06c40dc3d59538c3bff07aa2128ee814d9825c31f8b9abbf045e1ebb581bed` 1586 1587 ### Server Binaries 1588 1589 filename | sha512 hash 1590 -------- | ----------- 1591 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-server-linux-amd64.tar.gz) | `aed44ec5bc6bdaf41c20824a9841ee541bfd23362966f9326eca2cafbd03eca69325877784b8c9b058963dc5b8ba656b9da446513fd5eeed8133a783769acd16` 1592 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-server-linux-arm.tar.gz) | `e751cb675013183a70a8817dca0b5c456a1ccf075244b411317e813b5164ae710460a53e81191ded9d2ccaf284ae00304bfbe1b3d219b2a8d57761d733293409` 1593 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-server-linux-arm64.tar.gz) | `3aabb1a9bd07413d0740adfb638b0e5ca4cd4a58eda244c5ebc1ea01780e0b2863806c35792a0590069ef0cccc2665198afed1984d1c49f0726b75d4216609a4` 1594 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-server-linux-ppc64le.tar.gz) | `3c57e2b47b3b9ddc1039684840886877947141b1e4d31f909793678544fe92e10aa82a207936f0bcb3c657044c7b875f34f41f086ccb7f97a154499d01266f73` 1595 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-server-linux-s390x.tar.gz) | `fe418cfafc63cbccc1898086296e52026df27ff498753089792175ce0d41d889ca50a4eb5104a84b78a0a25d524dcdf5ec5eec8aa213d58178ae38411cfb58af` 1596 1597 ### Node Binaries 1598 1599 filename | sha512 hash 1600 -------- | ----------- 1601 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-node-linux-amd64.tar.gz) | `83b63d7e9d18fe35564105fd70629af9ba8f20112933b3ace92a48887702862e013ac3b3e144cfd44c8ead7c766df584766749af9d33a9aaa3808e370d3ea359` 1602 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-node-linux-arm.tar.gz) | `e4ac000be25ced9b308ec698da9702bd10a0a7183bfea9648500ab9d0879989c54328cb4fb51be545831d18676067d6b53e55cc49b14f55f35032a66dfa28806` 1603 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-node-linux-arm64.tar.gz) | `35fa5ec8a7fc001fe33abd89f8a446ce0ea2a011db27dc8ff544c2b199b065b19372afe95e3616f538347243d7599f29fb5cea1a46a3fcfdfe4a0f2a346683b2` 1604 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-node-linux-ppc64le.tar.gz) | `a5b938e9cdc39fc292269af4c3961a17b9bcdaf85b3c58db680f2d1a4fb088eb648efc268dfe3325aed96f6a7cdcff070dc7a3200cc169010ff2b402fae1a26a` 1605 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-node-linux-s390x.tar.gz) | `d8e2b5a945f881ddb0a25576a614d564ccef0ad4e93c84b30cdc57888e81e04932d798415a1a50cdfc6d2f857e1d027e2034ba9c40b5d8ed0009cfbdc8915e0f` 1606 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-beta.1/kubernetes-node-windows-amd64.tar.gz) | `426774466800ca11cae16821c7d92917b2ce138fcb3f6dca813ec0f060649d6a50187982aaac31fbb081604f5bd2c317616b6f211beb139f53192269194f012e` 1607 1608 ## Changelog since v1.14.0-alpha.3 1609 1610 ### Action Required 1611 1612 * Added cadvisor metric labels `pod` and `container` where `pod_name` and `container_name` are present to match instrumentation guidelines. ([#69099](https://github.com/kubernetes/kubernetes/pull/69099), [@ehashman](https://github.com/ehashman)) 1613 * Action required: any Prometheus queries that match `pod_name` and `container_name` labels (e.g. cadvisor or kubelet probe metrics) should be updated to use `pod` and `container` instead. `pod_name` and `container_name` labels will be present alongside `pod` and `container` labels for one transitional release and removed in the future. 1614 1615 ### Other notable changes 1616 1617 * fix parse devicePath issue on Azure Disk ([#74499](https://github.com/kubernetes/kubernetes/pull/74499), [@andyzhangx](https://github.com/andyzhangx)) 1618 * fix issue: fail to detach azure disk when there is server side error ([#74398](https://github.com/kubernetes/kubernetes/pull/74398), [@andyzhangx](https://github.com/andyzhangx)) 1619 * Allow Cinder volume limit to be configured from node too ([#74542](https://github.com/kubernetes/kubernetes/pull/74542), [@gnufied](https://github.com/gnufied)) 1620 * Fix subnet annotation checking for Azure internal loadbalancer ([#74498](https://github.com/kubernetes/kubernetes/pull/74498), [@feiskyer](https://github.com/feiskyer)) 1621 * Allow the kubelet to pass Windows GMSA credentials down to Docker ([#73726](https://github.com/kubernetes/kubernetes/pull/73726), [@wk8](https://github.com/wk8)) 1622 * PriorityClass in scheduling.k8s.io/v1beta1 and scheduling.k8s.io/v1alpha1 are deprecated by PriorityClass scheduling.k8s.io/v1 and will not be served starting in v1.17. ([#74465](https://github.com/kubernetes/kubernetes/pull/74465), [@bsalamat](https://github.com/bsalamat)) 1623 * kubeadm: fixed nil pointer dereference caused by a bug in url parsing ([#74454](https://github.com/kubernetes/kubernetes/pull/74454), [@bart0sh](https://github.com/bart0sh)) 1624 * Fix the unexpected NotReady status when Node's iops is full if the runtime is dockershim. ([#74389](https://github.com/kubernetes/kubernetes/pull/74389), [@answer1991](https://github.com/answer1991)) 1625 * Split up the mondo `kubernetes-test` tarball into `kubernetes-test-portable` and `kubernetes-test-{OS}-{ARCH}` tarballs. ([#74065](https://github.com/kubernetes/kubernetes/pull/74065), [@ixdy](https://github.com/ixdy)) 1626 * Move fluentd-elasticsearch addon images to community controlled location ([#73819](https://github.com/kubernetes/kubernetes/pull/73819), [@coffeepac](https://github.com/coffeepac)) 1627 * The PriorityClass API has been promoted to `scheduling.k8s.io/v1` with no changes. The `scheduling.k8s.io/v1beta1` version is now deprecated and will stop being served by default in v1.17. ([#73555](https://github.com/kubernetes/kubernetes/pull/73555), [@bsalamat](https://github.com/bsalamat)) 1628 * fix get azure accounts timeout issue when there is no out-bound IP ([#74191](https://github.com/kubernetes/kubernetes/pull/74191), [@andyzhangx](https://github.com/andyzhangx)) 1629 * fix mixed protocol issue for azure load balancer ([#74200](https://github.com/kubernetes/kubernetes/pull/74200), [@andyzhangx](https://github.com/andyzhangx)) 1630 * Don't update the Pod object after each scheduling attempt by adding a timestamp to the scheduling queue. ([#73700](https://github.com/kubernetes/kubernetes/pull/73700), [@denkensk](https://github.com/denkensk)) 1631 * kubeadm: remove local etcd members from the etcd cluster when kubeadm reset ([#74112](https://github.com/kubernetes/kubernetes/pull/74112), [@pytimer](https://github.com/pytimer)) 1632 * Fix keymutex issues which may crash in some platforms. ([#74348](https://github.com/kubernetes/kubernetes/pull/74348), [@danielqsj](https://github.com/danielqsj)) 1633 * Fixed scanning of failed iSCSI targets. ([#74306](https://github.com/kubernetes/kubernetes/pull/74306), [@jsafrane](https://github.com/jsafrane)) 1634 * kubeadm: Do not fail preflight checks when running on >= 5.0 Linux kernel ([#74355](https://github.com/kubernetes/kubernetes/pull/74355), [@brb](https://github.com/brb)) 1635 * Reduces the cache TTL for negative responses to 5s minimum. ([#74093](https://github.com/kubernetes/kubernetes/pull/74093), [@blakebarnett](https://github.com/blakebarnett)) 1636 * The Ingress API is now available via `networking.k8s.io/v1beta1`. `extensions/v1beta1` Ingress objects are deprecated and will no longer be served in v1.18. ([#74057](https://github.com/kubernetes/kubernetes/pull/74057), [@liggitt](https://github.com/liggitt)) 1637 * kubelet's --containerized flag will no longer be supported and will be removed in a future release ([#74267](https://github.com/kubernetes/kubernetes/pull/74267), [@dims](https://github.com/dims)) 1638 * Optimize scheduler cache snapshot algorithm to improve scheduling throughput. ([#74041](https://github.com/kubernetes/kubernetes/pull/74041), [@bsalamat](https://github.com/bsalamat)) 1639 * Extends the VolumeSubpathEnvExpansion alpha feature to support environment variable expansion ([#71351](https://github.com/kubernetes/kubernetes/pull/71351), [@kevtaylor](https://github.com/kevtaylor)) 1640 * Implements subPathExpr field for expanding environment variables into a subPath 1641 * The fields subPathExpr and subPath are mutually exclusive 1642 * Note: This is a breaking change from the previous version of this alpha feature 1643 * Added kube-proxy support for overlay networking and DSR in Windows and new flags for network-name, source-vip, and enable-dsr. ([#70896](https://github.com/kubernetes/kubernetes/pull/70896), [@ksubrmnn](https://github.com/ksubrmnn)) 1644 * StorageOS volume plugin updated to fix an issue where volume mount succeeds even if request to mount via StorageOS API fails. ([#69782](https://github.com/kubernetes/kubernetes/pull/69782), [@darkowlzz](https://github.com/darkowlzz)) 1645 * kubeadm: Allow to upload certificates required to join a new control-plane to kubeadm-certs secret using the flag `--experimental-upload-certs` on `init` or upload-certs phase. ([#73907](https://github.com/kubernetes/kubernetes/pull/73907), [@yagonobre](https://github.com/yagonobre)) 1646 * export query parameter is deprecated and will be removed in a future release ([#73783](https://github.com/kubernetes/kubernetes/pull/73783), [@deads2k](https://github.com/deads2k)) 1647 * e2e storage tests run faster and are easier to read ([#72434](https://github.com/kubernetes/kubernetes/pull/72434), [@pohly](https://github.com/pohly)) 1648 * kubectl: fix a bug where "describe" cannot obtain the event messages for a static pod ([#74156](https://github.com/kubernetes/kubernetes/pull/74156), [@gaorong](https://github.com/gaorong)) 1649 * windows: Ensure graceful termination when being run as windows service ([#73292](https://github.com/kubernetes/kubernetes/pull/73292), [@steffengy](https://github.com/steffengy)) 1650 * CoreDNS adds readinessProbe which prevents loadbalancing to unready pods, and also allows rolling updates to work as expected. ([#74137](https://github.com/kubernetes/kubernetes/pull/74137), [@rajansandeep](https://github.com/rajansandeep)) 1651 * Fixes use of webhook admission plugins with multi-version custom resources ([#74154](https://github.com/kubernetes/kubernetes/pull/74154), [@mbohlool](https://github.com/mbohlool)) 1652 * kubeadm no longer allows using v1alpha3 configs for anything else than converting them to v1beta1. ([#74025](https://github.com/kubernetes/kubernetes/pull/74025), [@rosti](https://github.com/rosti)) 1653 * Change kubelet metrics to conform metrics guidelines. ([#72470](https://github.com/kubernetes/kubernetes/pull/72470), [@danielqsj](https://github.com/danielqsj)) 1654 * The following metrics are deprecated, and will be removed in a future release: 1655 * `kubelet_pod_worker_latency_microseconds` 1656 * `kubelet_pod_start_latency_microseconds` 1657 * `kubelet_cgroup_manager_latency_microseconds` 1658 * `kubelet_pod_worker_start_latency_microseconds` 1659 * `kubelet_pleg_relist_latency_microseconds` 1660 * `kubelet_pleg_relist_interval_microseconds` 1661 * `kubelet_eviction_stats_age_microseconds` 1662 * `kubelet_runtime_operations` 1663 * `kubelet_runtime_operations_latency_microseconds` 1664 * `kubelet_runtime_operations_errors` 1665 * `kubelet_device_plugin_registration_count` 1666 * `kubelet_device_plugin_alloc_latency_microseconds` 1667 * Please convert to the following metrics: 1668 * `kubelet_pod_worker_duration_seconds` 1669 * `kubelet_pod_start_duration_seconds` 1670 * `kubelet_cgroup_manager_duration_seconds` 1671 * `kubelet_pod_worker_start_duration_seconds` 1672 * `kubelet_pleg_relist_duration_seconds` 1673 * `kubelet_pleg_relist_interval_seconds` 1674 * `kubelet_eviction_stats_age_seconds` 1675 * `kubelet_runtime_operations_total` 1676 * `kubelet_runtime_operations_duration_seconds` 1677 * `kubelet_runtime_operations_errors_total` 1678 * `kubelet_device_plugin_registration_total` 1679 * `kubelet_device_plugin_alloc_duration_seconds` 1680 * This change ensures that volumes get provisioned based on the zone information provided in allowedTopologies. ([#72731](https://github.com/kubernetes/kubernetes/pull/72731), [@skarthiksrinivas](https://github.com/skarthiksrinivas)) 1681 ``` 1682 Storage class spec: 1683 kind: StorageClass 1684 apiVersion: storage.k8s.io/v1 1685 metadata: 1686 name: fastpolicy1 1687 provisioner: kubernetes.io/vsphere-volume 1688 parameters: 1689 diskformat: zeroedthick 1690 storagePolicyName: vSAN Default Storage Policy 1691 allowedTopologies: 1692 - matchLabelExpressions: 1693 - key: failure-domain.beta.kubernetes.io/zone 1694 values: 1695 - zone1 1696 ``` 1697 * PV creation Logs: 1698 * I0109 11:17:52.321372 1 vsphere.go:1147] Starting to create a vSphere volume with volumeOptions: &{CapacityKB:1048576 Tags:map[kubernetes.io/created-for/pvc/namespace:default kubernetes.io/created-for/pvc/name:pvcsc-1-policy kubernetes.io/created-for/pv/name:pvc-34650c12-1400-11e9-aef4-005056804cc9] Name:kubernetes-dynamic-pvc-34650c12-1400-11e9-aef4-005056804cc9 DiskFormat:zeroedthick Datastore: VSANStorageProfileData: StoragePolicyName:vSAN Default Storage Policy StoragePolicyID: SCSIControllerType: Zone:[zone1]} 1699 * ... 1700 * I0109 11:17:59.430113 1 vsphere.go:1334] The canonical volume path for the newly created vSphere volume is "[vsanDatastore] 98db185c-6683-d8c7-bc55-0200435ec5da/kubernetes-dynamic-pvc-34650c12-1400-11e9-aef4-005056804cc9.vmdk" 1701 * Ran regression tests (no zone) and they passed. 1702 * vSphere cloud provider correctly retrieves the VM's UUID when running on Windows ([#71147](https://github.com/kubernetes/kubernetes/pull/71147), [@benmoss](https://github.com/benmoss)) 1703 * Re-issue Allocate grpc calls before starting a container that requests device-plugin resources if the cached state is missing. ([#73824](https://github.com/kubernetes/kubernetes/pull/73824), [@jiayingz](https://github.com/jiayingz)) 1704 * [CRI] Add a new field called `runtime_handler` into PodSandbox and PodSandboxStatus to track the RuntimeClass information of a pod. ([#73833](https://github.com/kubernetes/kubernetes/pull/73833), [@haiyanmeng](https://github.com/haiyanmeng)) 1705 * kubelet: OS and Arch information is now recorded in `kubernetes.io/os` and `kubernetes.io/arch` labels on Node objects. The previous labels (`beta.kubernetes.io/os` and `beta.kubernetes.io/arch`) are still recorded, but are deprecated and targeted for removal in 1.18. ([#73333](https://github.com/kubernetes/kubernetes/pull/73333), [@yujuhong](https://github.com/yujuhong)) 1706 * This change applies zone labels to vSphere Volumes automatically. The zone labels are visible on the PV: ([#72687](https://github.com/kubernetes/kubernetes/pull/72687), [@subramanian-neelakantan](https://github.com/subramanian-neelakantan)) 1707 * $ kubectl get pv --show-labels 1708 * NAME CAPACITY ACCESSMODES STATUS CLAIM REASON AGE LABELS 1709 * pv-abc 5Gi RWO Bound default/claim1 46s failure-domain.beta.kubernetes.io/region=VC1,failure-domain.beta.kubernetes.io/zone=cluster-1 1710 * fix smb remount issue on Windows ([#73661](https://github.com/kubernetes/kubernetes/pull/73661), [@andyzhangx](https://github.com/andyzhangx)) 1711 * Kubelet now tries to stop containers in unknown state once before restart or remove. ([#73802](https://github.com/kubernetes/kubernetes/pull/73802), [@Random-Liu](https://github.com/Random-Liu)) 1712 * Deprecate --export flag from kubectl get command. ([#73787](https://github.com/kubernetes/kubernetes/pull/73787), [@soltysh](https://github.com/soltysh)) 1713 * Breaking changes in client-go: ([#72214](https://github.com/kubernetes/kubernetes/pull/72214), [@caesarxuchao](https://github.com/caesarxuchao)) 1714 * The disk-cached discovery client is moved from k8s.io/client-go/discovery to k8s.io/client-go/discovery/cached/disk. 1715 * The memory-cached discovery client is moved from k8s.io/client-go/discovery/cached to k8s.io/client-go/discovery/cached/memory. 1716 * kubelet now accepts `pid=<number>` in the `--system-reserved` and `--kube-reserved` options to ensure that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively. Please reference `Kube Reserved` and `System Reserved` in `Reserve Compute Resources for System Daemons` in the Kubernetes documentation for general discussion of resource reservation. To utilize this functionality, you must set the feature gate `SupportNodePidsLimit=true` ([#73651](https://github.com/kubernetes/kubernetes/pull/73651), [@RobertKrawitz](https://github.com/RobertKrawitz)) 1717 * The apiserver, including both the kube-apiserver and apiservers built with the generic apiserver library, will now return 413 RequestEntityTooLarge error if a json patch contains more than 10,000 operations. ([#74000](https://github.com/kubernetes/kubernetes/pull/74000), [@caesarxuchao](https://github.com/caesarxuchao)) 1718 * kubeadm: allow the usage of --kubeconfig-dir and --config flags on kubeadm init ([#73998](https://github.com/kubernetes/kubernetes/pull/73998), [@yagonobre](https://github.com/yagonobre)) 1719 * when pleg channel is full, discard events and record its count ([#72709](https://github.com/kubernetes/kubernetes/pull/72709), [@changyaowei](https://github.com/changyaowei)) 1720 * Is ->It in line 6 ([#73898](https://github.com/kubernetes/kubernetes/pull/73898), [@xiezongzhe](https://github.com/xiezongzhe)) 1721 * Events reported for container creation, start, and stop now report the container name in the message and are more consistently formatted. ([#73892](https://github.com/kubernetes/kubernetes/pull/73892), [@smarterclayton](https://github.com/smarterclayton)) 1722 * `kubectl auth reconcile` now outputs details about what changes are being made ([#71564](https://github.com/kubernetes/kubernetes/pull/71564), [@liggitt](https://github.com/liggitt)) 1723 * kubeadm: fix a bug in the underlying library for diff related to characters like '%' ([#73941](https://github.com/kubernetes/kubernetes/pull/73941), [@neolit123](https://github.com/neolit123)) 1724 * kube-apiserver: a request body of a CREATE/UPDATE/PATCH/DELETE resource operation larger than 100 MB will return a 413 "request entity too large" error. ([#73805](https://github.com/kubernetes/kubernetes/pull/73805), [@caesarxuchao](https://github.com/caesarxuchao)) 1725 * Custom apiservers built with the latest apiserver library will have the 100MB limit on the body of resource requests as well. The limit can be altered via ServerRunOptions.MaxRequestBodyBytes. 1726 * The body size limit does not apply to subresources like pods/proxy that proxy request content to another server. 1727 * Kustomize is developed in its own repo https://github.com/kubernetes-sigs/kustomize ([#73033](https://github.com/kubernetes/kubernetes/pull/73033), [@Liujingfang1](https://github.com/Liujingfang1)) 1728 * This PR added a new subcommand `kustomize` in kubectl. 1729 * kubectl kustomize <somedir> has the same effect as kustomize build <somedir> 1730 * To build API resources from somedir with a kustomization.yaml file 1731 * kubectl kustomize <somedir> 1732 * This command can be piped to apply or delete 1733 * kubectl kustomize <somedir> | kubectl apply -f - 1734 * kubectl kustomize <somedir> | kubectl delete -f - 1735 * kubeadm: all master components are now exclusively relying on the `PriorityClassName` pod spec for annotating them as cluster critical components. Since `scheduler.alpha.kubernetes.io/critical-pod` annotation is no longer supported by Kubernetes 1.14 this annotation is no longer added to master components. ([#73857](https://github.com/kubernetes/kubernetes/pull/73857), [@ereslibre](https://github.com/ereslibre)) 1736 * Speedup kubectl by >10 when calling out to kube-apiserver for discovery information. ([#73345](https://github.com/kubernetes/kubernetes/pull/73345), [@sttts](https://github.com/sttts)) 1737 * kubeadm no longer dumps backtrace if it fails to remove the running containers on reset. ([#73951](https://github.com/kubernetes/kubernetes/pull/73951), [@rosti](https://github.com/rosti)) 1738 1739 1740 1741 # v1.14.0-alpha.3 1742 1743 [Documentation](https://docs.k8s.io) 1744 1745 ## Downloads for v1.14.0-alpha.3 1746 1747 1748 filename | sha512 hash 1749 -------- | ----------- 1750 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes.tar.gz) | `5060dcf689dad4e19da5029eb8fc3060a4b2bad988fddff438d0703a45c02481bcfbc15f45d2855f4fd5e9eb43847400ebb25dce19e24f0e0e194a7f57176ce5` 1751 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-src.tar.gz) | `754c948b5d25b01f211866d473257be5fb576b4b97703eb6fc08679d6525e1f53195a450f3f47b77fabb92bf058583b66230959197b5bcf72528e54ccb349c07` 1752 1753 ### Client Binaries 1754 1755 filename | sha512 hash 1756 -------- | ----------- 1757 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-darwin-386.tar.gz) | `5bd74dfc86bacf89d6b05d541e13bf390216039a42cc90fef2b248820acd84f56a445ec66d52497ff77e1af47455f285c993cd1d44cc3050996189bd328ea2be` 1758 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-darwin-amd64.tar.gz) | `34e16661d66d337083583dfb478756ec8cc664d7cfc2dd1817bf1da03cdc380668be9df9f178b5fd5ccab5014e6686f83b9fee6192fbf77d2298d397e872a893` 1759 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-linux-386.tar.gz) | `15f99e85bcc95f7b8e1b4c6ecc23de36e89a54108003db926e97ec2e7253f363f6ed85e39a47305dbccf596f72e88edd7bcda6d528919da9c0b81541f58506d4` 1760 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-linux-amd64.tar.gz) | `2e61cf9b776150c4f1830d068ffee9701cb04979152ed6b62fc1bf53163e6194029a4f75536e7fda71c3dfce1de285f425bde342a4efdd1f7bf973f105750ac4` 1761 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-linux-arm.tar.gz) | `67fb3805bb1b4a77f6603fbde9bd1d26e179de1a594c85618aa7b17be6abc510a9a0cd499ef4fe974574cf73b364da641121f21864c8472d713eec76e4c52bca` 1762 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-linux-arm64.tar.gz) | `28930dc384b51051081a52874bc4d6dafa3c992dfa214b977ef711de2c2bc3f90bdaa6243bded1e750997fec04b8ffb910db21c266e47e09426c4dbaf916a64d` 1763 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-linux-ppc64le.tar.gz) | `f59eda797a57961d52fe67ba8b25a3a10267f9ce46029ed2140ef4b02615ba9944bd83d7a6e7874c7268a09a3422858b9b0c31f861941ef8be126c594fc3a7cc` 1764 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-linux-s390x.tar.gz) | `c56bfb64e55cf95251157a8229a3e94310b2c46bb1c1250050893873e3112578978c1f8e29fa56fac63e2aa8a6382523ac34baf6dd523fe0919f8d702521a564` 1765 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-windows-386.tar.gz) | `e49a00fbe600892dc5eed0bc21bac64806da65280c818ca79b5e8adbed7fd5ecebb6b647cb9b89ac862257995145b2397996122eefb3c8d127d857c89c29c9ae` 1766 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-client-windows-amd64.tar.gz) | `797e20969ed4935adcbc80ccbcd72ec5aa697e70b0d071eceefc6dbacea69aff9f6660e7eefad6661ace0afb66067c4ffaa4f6bc82e8b081b57811ab0abde218` 1767 1768 ### Server Binaries 1769 1770 filename | sha512 hash 1771 -------- | ----------- 1772 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-server-linux-amd64.tar.gz) | `eddfc9afd7337475c3865443170d1425dcf4a87d981555871a69bcf132e73d99b1ffa08a00490b30c60232f47bbeca4ad6253cf7e1dad44797b4af044dbdbef4` 1773 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-server-linux-arm.tar.gz) | `dc85cd3a039cc0516beb19018c8378f3b7b88fa2edb8fa1476305e89eb7c64fef2d938bd48fd257ea8e690f7d84a69e9784a42aabed35e83ea7362c60773ba67` 1774 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-server-linux-arm64.tar.gz) | `d7c3a72abaa4c3e3243f8b4b3a8adb8be2758e0f883423ea62d2c61b2081464a8976ad43ea0640a7e453aa4d389e3ea2d6d1baedf3b50e1171eca6e49cd087fe` 1775 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-server-linux-ppc64le.tar.gz) | `b268a94eb056eea8bdf4d5739dec430f75a6a6b3c18e30df68d970c3566b3e4a638b3577f6219596ae54eac740628a7ebfecb0772645e6d960f790235e1d62c7` 1776 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-server-linux-s390x.tar.gz) | `f4cfd8d2faacdd1f0065f9e0f4f8d0db7bd8f438f812f70a07f4cb5272ae9bed3ec876b3cbaf2f2a71e65e4de725e1dc0829b43f60f43c9e43656ac928657d5e` 1777 1778 ### Node Binaries 1779 1780 filename | sha512 hash 1781 -------- | ----------- 1782 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-node-linux-amd64.tar.gz) | `7040ee3c032ec4fe14530c3e47ee53d731acb947b06e2d560cbcd0e7e513142c0f300302059aaef03e24311946a9c59b576948eec9b520e2367f28fc4f80226c` 1783 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-node-linux-arm.tar.gz) | `3d32e5243d1c65bce573cfb0f60d643ef3fc684a15551dbc8c3d5435e6854ff104c46c77b0b8708d9c661d52f7865a197ea758f0c17e1ed991993674929ea75e` 1784 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-node-linux-arm64.tar.gz) | `d3a17027fa1c057528422b35e32260f5b7c7246400df595f0ebda5d150456d4388129b1ead4229f98f2b461ff9e85382a7da0d682541844a3c06f0aebe0469b6` 1785 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-node-linux-ppc64le.tar.gz) | `89ed1f5093b49ab9d58d7a70089e881bf388f3316cb2607fa18e3bf072aff3d27aabe99124334774e63decb67349eb82f33ea509b56a72a51e1443c3352b4558` 1786 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-node-linux-s390x.tar.gz) | `755a60824a9b8c4090a791d332e410692708ecece90e37388f58eb2c7ddddea6b859fefcc5a53ec3d275fee0a355086f4446ae8e85482a668d248cca9f5e503c` 1787 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.3/kubernetes-node-windows-amd64.tar.gz) | `c71d8055d89e535771f345e0f673da021915a7a82c75951855ba2574a4250c8a57d0636b4ec9bba209edde8edef30098c6dec2f80403cd46139bb88d814c3751` 1788 1789 ## Changelog since v1.14.0-alpha.2 1790 1791 ### Action Required 1792 1793 * The --storage-versions flag of kube-apiserver is removed. The storage versions will always be the default value built-in the kube-apiserver binary. ([#67678](https://github.com/kubernetes/kubernetes/pull/67678), [@caesarxuchao](https://github.com/caesarxuchao)) 1794 1795 ### Other notable changes 1796 1797 * fix [#73264](https://github.com/kubernetes/kubernetes/pull/73264) cpuPeriod was not reset, but used as set via flag, although it was disabled via alpha gate ([#73342](https://github.com/kubernetes/kubernetes/pull/73342), [@szuecs](https://github.com/szuecs)) 1798 * Update kubelet CLI summary documentation and generated Webpage ([#73256](https://github.com/kubernetes/kubernetes/pull/73256), [@deitch](https://github.com/deitch)) 1799 * Considerably reduced the CPU load in kube-apiserver while aggregating OpenAPI specifications from aggregated API servers. ([#71223](https://github.com/kubernetes/kubernetes/pull/71223), [@sttts](https://github.com/sttts)) 1800 * kubeadm: add a preflight check that throws a warning if the cgroup driver for Docker on Linux is not "systemd" as per the k8s.io CRI installation guide. ([#73837](https://github.com/kubernetes/kubernetes/pull/73837), [@neolit123](https://github.com/neolit123)) 1801 * Kubelet: add usageNanoCores from CRI stats provider ([#73659](https://github.com/kubernetes/kubernetes/pull/73659), [@feiskyer](https://github.com/feiskyer)) 1802 * Fix watch to not send the same set of events multiple times causing watcher to go back in time ([#73845](https://github.com/kubernetes/kubernetes/pull/73845), [@wojtek-t](https://github.com/wojtek-t)) 1803 * `system:kube-controller-manager` and `system:kube-scheduler` users are now permitted to perform delegated authentication/authorization checks by default RBAC policy ([#72491](https://github.com/kubernetes/kubernetes/pull/72491), [@liggitt](https://github.com/liggitt)) 1804 * Prevent AWS Network Load Balancer security groups ingress rules to be deleted by ensuring target groups are tagged. ([#73594](https://github.com/kubernetes/kubernetes/pull/73594), [@masterzen](https://github.com/masterzen)) 1805 * Set a low oom_score_adj for containers in pods with system-critical priorities ([#73758](https://github.com/kubernetes/kubernetes/pull/73758), [@sjenning](https://github.com/sjenning)) 1806 * Ensure directories on volumes are group-executable when using fsGroup ([#73533](https://github.com/kubernetes/kubernetes/pull/73533), [@mxey](https://github.com/mxey)) 1807 * kube-apiserver now only aggregates openapi schemas from `/openapi/v2` endpoints of aggregated API servers. The fallback to aggregate from `/swagger.json` has been removed. Ensure aggregated API servers provide schema information via `/openapi/v2` (available since v1.10). ([#73441](https://github.com/kubernetes/kubernetes/pull/73441), [@roycaihw](https://github.com/roycaihw)) 1808 * Change docker metrics to conform metrics guidelines and using histogram for better aggregation. ([#72323](https://github.com/kubernetes/kubernetes/pull/72323), [@danielqsj](https://github.com/danielqsj)) 1809 * The following metrics are deprecated, and will be removed in a future release: 1810 * `docker_operations` 1811 * `docker_operations_latency_microseconds` 1812 * `docker_operations_errors` 1813 * `docker_operations_timeout` 1814 * `network_plugin_operations_latency_microseconds` 1815 * Please convert to the following metrics: 1816 * `docker_operations_total` 1817 * `docker_operations_latency_seconds` 1818 * `docker_operations_errors_total` 1819 * `docker_operations_timeout_total` 1820 * `network_plugin_operations_latency_seconds` 1821 * `kubectl delete --all-namespaces` is a recognized flag. ([#73716](https://github.com/kubernetes/kubernetes/pull/73716), [@deads2k](https://github.com/deads2k)) 1822 * MAC Address filter has been fixed in vSphere Cloud Provider, it no longer ignores `00:1c:14` and `00:05:69` prefixes ([#73721](https://github.com/kubernetes/kubernetes/pull/73721), [@frapposelli](https://github.com/frapposelli)) 1823 * Add kubelet_node_name metrics. ([#72910](https://github.com/kubernetes/kubernetes/pull/72910), [@danielqsj](https://github.com/danielqsj)) 1824 * The HugePages feature gate has graduated to GA, and can no longer be disabled. The feature gate will be removed in v1.16 ([#72785](https://github.com/kubernetes/kubernetes/pull/72785), [@derekwaynecarr](https://github.com/derekwaynecarr)) 1825 * Fix a bug that aggregated openapi spec may override swagger securityDefinitions and swagger info in kube-apiserver ([#73484](https://github.com/kubernetes/kubernetes/pull/73484), [@roycaihw](https://github.com/roycaihw)) 1826 * Fixes a bug that prevented deletion of dynamically provisioned volumes in Quobyte backends. ([#68925](https://github.com/kubernetes/kubernetes/pull/68925), [@casusbelli](https://github.com/casusbelli)) 1827 * error messages returned in authentication webhook status responses are now correctly included in the apiserver log ([#73595](https://github.com/kubernetes/kubernetes/pull/73595), [@liggitt](https://github.com/liggitt)) 1828 * kubeadm: `kubeadm alpha preflight` and `kubeadm alpha preflight node` are removed; you can now use `kubeadm join phase preflight` ([#73718](https://github.com/kubernetes/kubernetes/pull/73718), [@fabriziopandini](https://github.com/fabriziopandini)) 1829 * kube-apiserver: the deprecated `repair-malformed-updates` has been removed ([#73663](https://github.com/kubernetes/kubernetes/pull/73663), [@danielqsj](https://github.com/danielqsj)) 1830 * e2e.test now rejects unknown --provider values instead of merely warning about them. An empty provider name is not accepted anymore and was replaced by "skeleton" (= a provider with no special behavior). ([#73402](https://github.com/kubernetes/kubernetes/pull/73402), [@pohly](https://github.com/pohly)) 1831 * Updated AWS SDK to v1.16.26 for ECR PrivateLink support ([#73435](https://github.com/kubernetes/kubernetes/pull/73435), [@micahhausler](https://github.com/micahhausler)) 1832 * Expand kubectl wait to work with more types of selectors. ([#71746](https://github.com/kubernetes/kubernetes/pull/71746), [@rctl](https://github.com/rctl)) 1833 * The CustomPodDNS feature gate has graduated to GA, and can no longer be disabled. The feature gate will be removed in v1.16 ([#72832](https://github.com/kubernetes/kubernetes/pull/72832), [@MrHohn](https://github.com/MrHohn)) 1834 * The `rules` field in RBAC Role and ClusterRole objects is now correctly reported as optional in the openapi schema. ([#73250](https://github.com/kubernetes/kubernetes/pull/73250), [@liggitt](https://github.com/liggitt)) 1835 * AWS ELB health checks will now use HTTPS/SSL protocol for HTTPS/SSL backends. ([#70309](https://github.com/kubernetes/kubernetes/pull/70309), [@2rs2ts](https://github.com/2rs2ts)) 1836 * kubeadm reset: fixed crash caused by absence of a configuration file ([#73636](https://github.com/kubernetes/kubernetes/pull/73636), [@bart0sh](https://github.com/bart0sh)) 1837 * CoreDNS is now version 1.3.1 ([#73610](https://github.com/kubernetes/kubernetes/pull/73610), [@rajansandeep](https://github.com/rajansandeep)) 1838 * A new `k8s_external` plugin that allows external zones to point to Kubernetes in-cluster services. 1839 * CoreDNS now checks if a zone transfer is allowed. Also allow a TTL of 0 to avoid caching in the cache plugin. 1840 * TTL is also applied to negative responses (NXDOMAIN, etc). 1841 1842 * Missing directories listed in a user's PATH are no longer considered errors and are instead logged by the "kubectl plugin list" command when listing available plugins. ([#73542](https://github.com/kubernetes/kubernetes/pull/73542), [@juanvallejo](https://github.com/juanvallejo)) 1843 * remove kubelet flag '--experimental-fail-swap-on' (deprecated in v1.8) ([#69552](https://github.com/kubernetes/kubernetes/pull/69552), [@Pingan2017](https://github.com/Pingan2017)) 1844 * Introduced support for Windows nodes into the cluster bringup scripts for GCE. ([#73442](https://github.com/kubernetes/kubernetes/pull/73442), [@pjh](https://github.com/pjh)) 1845 * Now users could get object info like: ([#73063](https://github.com/kubernetes/kubernetes/pull/73063), [@WanLinghao](https://github.com/WanLinghao)) 1846 * a. kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[0:3].name 1847 * b. kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[-2:].name 1848 * scheduler: use incremental scheduling cycle in PriorityQueue to put all in-flight unschedulable pods back to active queue if we received move request ([#73309](https://github.com/kubernetes/kubernetes/pull/73309), [@cofyc](https://github.com/cofyc)) 1849 * fixes an error processing watch events when running skewed apiservers ([#73482](https://github.com/kubernetes/kubernetes/pull/73482), [@liggitt](https://github.com/liggitt)) 1850 * Prometheus metrics for crd_autoregister, crd_finalizer and crd_naming_condition_controller are exported. ([#71767](https://github.com/kubernetes/kubernetes/pull/71767), [@roycaihw](https://github.com/roycaihw)) 1851 * Adds deleting pods created by DaemonSet assigned to not existing nodes. ([#73401](https://github.com/kubernetes/kubernetes/pull/73401), [@krzysztof-jastrzebski](https://github.com/krzysztof-jastrzebski)) 1852 * Graduate Pod Priority and Preemption to GA. ([#73498](https://github.com/kubernetes/kubernetes/pull/73498), [@bsalamat](https://github.com/bsalamat)) 1853 * Adds configuration for AWS endpoint fine control: ([#72245](https://github.com/kubernetes/kubernetes/pull/72245), [@ampsingram](https://github.com/ampsingram)) 1854 * OverrideEndpoints bool Set to true to allow custom endpoints 1855 * ServiceDelimiter string Delimiter to use to separate overridden services (multiple services) Defaults to "&" 1856 * ServicenameDelimiter string Delimiter to use to separate servicename from its configuration parameters Defaults "|" 1857 * OverrideSeparator string Delimiter to use to separate region of occurrence, url and signing region for each override Defaults to "," 1858 * ServiceOverrides string example: s3|region1, https://s3.foo.bar, some signing_region & ec2|region2, https://ec2.foo.bar, signing_region 1859 * The CoreDNS configuration now has the forward plugin for proxy in the default configuration instead of the proxy plugin. ([#73267](https://github.com/kubernetes/kubernetes/pull/73267), [@rajansandeep](https://github.com/rajansandeep)) 1860 * Fixed a bug that caused PV allocation on non-English vSphere installations to fail ([#73115](https://github.com/kubernetes/kubernetes/pull/73115), [@alvaroaleman](https://github.com/alvaroaleman)) 1861 1862 1863 1864 # v1.14.0-alpha.2 1865 1866 [Documentation](https://docs.k8s.io) 1867 1868 ## Downloads for v1.14.0-alpha.2 1869 1870 1871 filename | sha512 hash 1872 -------- | ----------- 1873 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes.tar.gz) | `1330e4421b61f6b1e6e4dee276d4742754bd3dd4493508d67ebb4445065277c619c4da8b4835febf0b2cdcf9e75fce96de1c1d99998904bae2bb794a453693f2` 1874 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-src.tar.gz) | `352c043bebf13a616441c920f3eec80d3f02f111d8488c31aa903e1483bce6d1fbe7472208f64730142960c8f778ab921ef7b654540a3ec09e53bd7e644521bd` 1875 1876 ### Client Binaries 1877 1878 filename | sha512 hash 1879 -------- | ----------- 1880 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-darwin-386.tar.gz) | `ee5aba4efce323167e6d897a2ff6962a240e466333bcae9390be2c8521c6da50ac2cb6139510b693aad49d6393b97a2118ed1fe4f999dd08bdca6d875d25f804` 1881 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-darwin-amd64.tar.gz) | `4b5c0b340322956a8d096c595124a765ac318d0eb460d6320218f2470e22d88221a0a9f1f93d5f3075f1c36b18c7041ee2fcb32e0f9c94d9f79bc3fd3005e68e` 1882 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-linux-386.tar.gz) | `7a5bfe68dd58c8478746a410872b615daf8abb9a78754140fb4d014a0c9177a87859ac046f56f5743fb97a9881abc2cf48c3e51aa02c8a86a754bf2cc59edb54` 1883 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-linux-amd64.tar.gz) | `c3139f58070241f2da815f701af3c0bd0ea4fdec1fe54bb859bd11237ac9b75ecb01b62ac1c7a459a4dd79696412c6d2f8cbd492fd062a790ceadd3dcc9b07fd` 1884 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-linux-arm.tar.gz) | `9d96d2e1e11aa61e2c3a5f4f27c18866feae9833b6ee70b15f5cdb5f992849dc1f79821af856b467487092a21a447231fb9c4de6ee6f17defed3cfa16d35b4c6` 1885 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-linux-arm64.tar.gz) | `7b4dd825cf9f217c18b28976a3faa94f0bd4868e541e5be7d57cd770e2b163c6daddf12e5f9ad51d92abde794a444f2a20bf582a30f03c39e60186d356030a2d` 1886 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-linux-ppc64le.tar.gz) | `490638e250c24b6bad8b67358fd7890f7a2f6456ae8ffe537c28bb5b3ce7abc591e6fecbddd6744f0f6c0e24b9f44c31f7ca1f7ebfc3c0d17a96fe8cf27b8548` 1887 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-linux-s390x.tar.gz) | `9dd8c3361eda15dd1594066c55b79cb9a34578c225b2b48647cd5b34619cf23106b845ee25b80d979f8b69e8733148842177500dc48989177b6944677f071f1c` 1888 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-windows-386.tar.gz) | `d624b8aead053201765b713d337528be82a71328ee3dd569f556868ceeb4904e64584892a016d247608fc4521c00ead7aed5d973b1206caa2d00406532d5b8b4` 1889 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-client-windows-amd64.tar.gz) | `a1cf8c67984dd4eb4610fa05d27fe9e9e4123159f933e3986e9db835b9cf136962168f0003071001e01e2c1831804ba0a366f2495741aa60a41587a69c09cb62` 1890 1891 ### Server Binaries 1892 1893 filename | sha512 hash 1894 -------- | ----------- 1895 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-server-linux-amd64.tar.gz) | `b93982b56371994c540cd11e6bc21808279340617164992c10f30d8e6ae4d5e270e41c1edc0625d3458a18944ec7aa8c273acbbcd718d60b6cacbc24220c42ac` 1896 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-server-linux-arm.tar.gz) | `bfd76c6b26e5927166d776f6110b97ee36c1d63ad39e2d18899f3e428ebb0f9615bb677ac8e9bcc1864c72a40efd71e1314fe6d137f9c6e54f720270929e3f46` 1897 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-server-linux-arm64.tar.gz) | `6721dec0df9466cd6c056160c73d598296cebb0af9259eb21b693abb8708901bc8bc30e11815e14d00d6eb12b8bb90b699e3119b922da855e2c411bdf229d6e5` 1898 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-server-linux-ppc64le.tar.gz) | `f8cd307db8141d989ae1218dd2b438bc9cee017d533b1451d2345f9689c451fdb080acd1b9b2f535ed04017e44b81a0585072e7d58a9d201a0ec28fd09df0a6f` 1899 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-server-linux-s390x.tar.gz) | `de7514bbd87a1b363e1bc7787f37d5ea10faac4afe7c5163c23c4df16781aa77570ec553bc4f4b6094166c1fcfc3c431f13e51ffa32f7ea2849e76ec0151ea35` 1900 1901 ### Node Binaries 1902 1903 filename | sha512 hash 1904 -------- | ----------- 1905 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-node-linux-amd64.tar.gz) | `8c37fd2fe6232d2c148e23df021b8b5347136263399932bcdff0c7a0186f3145de9ede4936b14de7484cc6db9241517d79b5306c380ed374396882900b63e912` 1906 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-node-linux-arm.tar.gz) | `389e4e77ab9e62968a25b8f4e146a2c3fbb3db2e60e051922edf6395c26cc5380e5a77bf67022339d6ebfe9abd714636d77510bbc42924b4265fdb245fae08c9` 1907 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-node-linux-arm64.tar.gz) | `7efc32dfeefcef7f860913c25431bd891a435e92cb8d5a95f8deca1a82aa899a007d4b19134493694a4bccb5564867488634a780c128f0cf82c61d98afa889f5` 1908 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-node-linux-ppc64le.tar.gz) | `da30c03bca4b81d810a7df006db02333dea87e336d6cdca9c93392e01c7e43bf4902c969efa7fa53e8a70a0e863b403ec26b87bd38226b8b9f98777ddb0051a0` 1909 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-node-linux-s390x.tar.gz) | `cce43b7f0350b9e5a77ea703225adb9714ef022d176db5b99a0327937d19021d7a8e93ef1169389fd53b895bb98725d23c7565ef80afdd17596c26daf41eeeac` 1910 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.2/kubernetes-node-windows-amd64.tar.gz) | `d3accf522d80cbfb3d03e9eaa60a09767ba11e88a8a5b44a629192a7c6916b1fb3440f022a5ffc4ea78f3595f254a42f028dd428d117360091cd0c747ec39eb5` 1911 1912 ## Changelog since v1.14.0-alpha.1 1913 1914 ### Action Required 1915 1916 * Promote ValidateProxyRedirects to Beta, and enable by default. This feature restricts redirect following from the apiserver to same-host redirects. ([#72552](https://github.com/kubernetes/kubernetes/pull/72552), [@tallclair](https://github.com/tallclair)) 1917 * ACTION REQUIRED: If nodes are configured to respond to CRI streaming requests on a different host interface than what the apiserver makes requests on (only the case if not using the built-in dockershim & setting the kubelet flag `--redirect-container-streaming=true`), then these requests will be broken. In that case, the feature can be temporarily disabled until the node configuration is corrected. We suggest setting `--redirect-container-streaming=false` on the kubelet to avoid issues. 1918 1919 ### Other notable changes 1920 1921 * Added alpha field storageVersionHash to the discovery document for each resource. Its value must be treated as opaque by clients. Only equality comparison on the value is valid. ([#73191](https://github.com/kubernetes/kubernetes/pull/73191), [@caesarxuchao](https://github.com/caesarxuchao)) 1922 * Fix admission metrics in seconds. ([#72343](https://github.com/kubernetes/kubernetes/pull/72343), [@danielqsj](https://github.com/danielqsj)) 1923 * Add metrics `*_admission_latencies_milliseconds` and `*_admission_latencies_milliseconds_summary` for backward compatible, but will be removed in a future release. 1924 * Pod eviction now honors graceful deletion by default if no delete options are provided in the eviction request ([#72730](https://github.com/kubernetes/kubernetes/pull/72730), [@liggitt](https://github.com/liggitt)) 1925 * Update to go1.11.5 ([#73326](https://github.com/kubernetes/kubernetes/pull/73326), [@ixdy](https://github.com/ixdy)) 1926 * Change proxy metrics to conform metrics guidelines. ([#72334](https://github.com/kubernetes/kubernetes/pull/72334), [@danielqsj](https://github.com/danielqsj)) 1927 * The metrics `sync_proxy_rules_latency_microseconds` is deprecated, and will be removed in a future release, please convert to metrics`sync_proxy_rules_latency_seconds`. 1928 * Add network stats for Windows nodes and pods. ([#70121](https://github.com/kubernetes/kubernetes/pull/70121), [@feiskyer](https://github.com/feiskyer)) 1929 * kubeadm: When certificates are present joining a new control plane make sure that they match at least the required SANs ([#73093](https://github.com/kubernetes/kubernetes/pull/73093), [@ereslibre](https://github.com/ereslibre)) 1930 * A new `TaintNodesByCondition` admission plugin taints newly created Node objects as "not ready", to fix a race condition that could cause pods to be scheduled on new nodes before their taints were updated to accurately reflect their reported conditions. This admission plugin is enabled by default if the `TaintNodesByCondition` feature is enabled. ([#73097](https://github.com/kubernetes/kubernetes/pull/73097), [@bsalamat](https://github.com/bsalamat)) 1931 * kube-addon-manager was updated to v9.0, and now uses kubectl v1.13.2 and prunes workload resources via the apps/v1 API ([#72978](https://github.com/kubernetes/kubernetes/pull/72978), [@liggitt](https://github.com/liggitt)) 1932 * When a watch is closed by an HTTP2 load balancer and we are told to go away, skip printing the message to stderr by default. ([#73277](https://github.com/kubernetes/kubernetes/pull/73277), [@smarterclayton](https://github.com/smarterclayton)) 1933 * If you are running the cloud-controller-manager and you have the `pvlabel.kubernetes.io` alpha Initializer enabled, you must now enable PersistentVolume labeling using the `PersistentVolumeLabel` admission controller instead. You can do this by adding `PersistentVolumeLabel` in the `--enable-admission-plugins` kube-apiserver flag. ([#73102](https://github.com/kubernetes/kubernetes/pull/73102), [@andrewsykim](https://github.com/andrewsykim)) 1934 * The alpha Initializers feature, `admissionregistration.k8s.io/v1alpha1` API version, `Initializers` admission plugin, and use of the `metadata.initializers` API field have been removed. Discontinue use of the alpha feature and delete any existing `InitializerConfiguration` API objects before upgrading. The `metadata.initializers` field will be removed in a future release. ([#72972](https://github.com/kubernetes/kubernetes/pull/72972), [@liggitt](https://github.com/liggitt)) 1935 * Scale max-inflight limits together with master VM sizes. ([#73268](https://github.com/kubernetes/kubernetes/pull/73268), [@wojtek-t](https://github.com/wojtek-t)) 1936 * kubectl supports copying files with wild card ([#72641](https://github.com/kubernetes/kubernetes/pull/72641), [@dixudx](https://github.com/dixudx)) 1937 * kubeadm: add back `--cert-dir` option for `kubeadm init phase certs sa` ([#73239](https://github.com/kubernetes/kubernetes/pull/73239), [@mattkelly](https://github.com/mattkelly)) 1938 * Remove deprecated args '--show-all' ([#69255](https://github.com/kubernetes/kubernetes/pull/69255), [@Pingan2017](https://github.com/Pingan2017)) 1939 * As per deprecation policy in https://kubernetes.io/docs/reference/using-api/deprecation-policy/ ([#73001](https://github.com/kubernetes/kubernetes/pull/73001), [@shivnagarajan](https://github.com/shivnagarajan)) 1940 * the taints "node.alpha.kubernetes.io/notReady" and "node.alpha.kubernetes.io/unreachable". are no 1941 * longer supported or adjusted. These uses should be replaced with "node.kubernetes.io/not-ready" 1942 * and "node.kubernetes.io/unreachable" respectively instead. 1943 * The /swagger.json and /swagger-2.0.0.pb-v1 schema documents, deprecated since v1.10, have been removed in favor of `/openapi/v2` ([#73148](https://github.com/kubernetes/kubernetes/pull/73148), [@liggitt](https://github.com/liggitt)) 1944 * CoreDNS is only officially supported on Linux at this time. As such, when kubeadm is used to deploy this component into your kubernetes cluster, it will be restricted (using nodeSelectors) to run only on nodes with that operating system. This ensures that in clusters which include Windows nodes, the scheduler will not ever attempt to place CoreDNS pods on these machines, reducing setup latency and enhancing initial cluster stability. ([#69940](https://github.com/kubernetes/kubernetes/pull/69940), [@MarcPow](https://github.com/MarcPow)) 1945 * kubeadm now attempts to detect an installed CRI by its usual domain socket, so that --cri-socket can be omitted from the command line if Docker is not used and there is a single CRI installed. ([#69366](https://github.com/kubernetes/kubernetes/pull/69366), [@rosti](https://github.com/rosti)) 1946 * scheduler: makes pod less racing so as to be put back into activeQ properly ([#73078](https://github.com/kubernetes/kubernetes/pull/73078), [@Huang-Wei](https://github.com/Huang-Wei)) 1947 * jsonpath expressions containing `[start:end:step]` slice are now evaluated correctly ([#73149](https://github.com/kubernetes/kubernetes/pull/73149), [@liggitt](https://github.com/liggitt)) 1948 * metadata.deletionTimestamp is no longer moved into the future when issuing repeated DELETE requests against a resource containing a finalizer. ([#73138](https://github.com/kubernetes/kubernetes/pull/73138), [@liggitt](https://github.com/liggitt)) 1949 * The "kubectl api-resources" command will no longer fail to display any resources on a single failure ([#73035](https://github.com/kubernetes/kubernetes/pull/73035), [@juanvallejo](https://github.com/juanvallejo)) 1950 * e2e tests that require SSH may be used against clusters that have nodes without external IP addresses by setting the environment variable `KUBE_SSH_BASTION` to the `host:port` of a machine that is allowed to SSH to those nodes. The same private key that the test would use is used for the bastion host. The test connects to the bastion and then tunnels another SSH connection to the node. ([#72286](https://github.com/kubernetes/kubernetes/pull/72286), [@smarterclayton](https://github.com/smarterclayton)) 1951 * kubeadm: explicitly wait for `etcd` to have grown when joining a new control plane ([#72984](https://github.com/kubernetes/kubernetes/pull/72984), [@ereslibre](https://github.com/ereslibre)) 1952 * Install CSINodeInfo and CSIDriver CRDs in the local cluster. ([#72584](https://github.com/kubernetes/kubernetes/pull/72584), [@xing-yang](https://github.com/xing-yang)) 1953 * kubectl loads config file once and uses persistent client config ([#71117](https://github.com/kubernetes/kubernetes/pull/71117), [@dixudx](https://github.com/dixudx)) 1954 * remove stale OutOfDisk condition from kubelet side ([#72507](https://github.com/kubernetes/kubernetes/pull/72507), [@dixudx](https://github.com/dixudx)) 1955 * Node OS/arch labels are promoted to GA ([#73048](https://github.com/kubernetes/kubernetes/pull/73048), [@yujuhong](https://github.com/yujuhong)) 1956 * Fix graceful apiserver shutdown to not drop outgoing bytes before the process terminates. ([#72970](https://github.com/kubernetes/kubernetes/pull/72970), [@sttts](https://github.com/sttts)) 1957 * Change apiserver metrics to conform metrics guidelines. ([#72336](https://github.com/kubernetes/kubernetes/pull/72336), [@danielqsj](https://github.com/danielqsj)) 1958 * The following metrics are deprecated, and will be removed in a future release: 1959 * `apiserver_request_count` 1960 * `apiserver_request_latencies` 1961 * `apiserver_request_latencies_summary` 1962 * `apiserver_dropped_requests` 1963 * `etcd_helper_cache_hit_count` 1964 * `etcd_helper_cache_miss_count` 1965 * `etcd_helper_cache_entry_count` 1966 * `etcd_request_cache_get_latencies_summary` 1967 * `etcd_request_cache_add_latencies_summary` 1968 * `etcd_request_latencies_summary` 1969 * `transformation_latencies_microseconds ` 1970 * `data_key_generation_latencies_microseconds` 1971 * Please convert to the following metrics: 1972 * `apiserver_request_total` 1973 * `apiserver_request_latency_seconds` 1974 * `apiserver_dropped_requests_total` 1975 * `etcd_helper_cache_hit_total` 1976 * `etcd_helper_cache_miss_total` 1977 * `etcd_helper_cache_entry_total` 1978 * `etcd_request_cache_get_latency_seconds` 1979 * `etcd_request_cache_add_latency_seconds` 1980 * `etcd_request_latency_seconds` 1981 * `transformation_latencies_seconds` 1982 * `data_key_generation_latencies_seconds` 1983 * acquire lock before operating unschedulablepodsmap ([#73022](https://github.com/kubernetes/kubernetes/pull/73022), [@denkensk](https://github.com/denkensk)) 1984 * Print `SizeLimit` of `EmptyDir` in `kubectl describe pod` outputs. ([#69279](https://github.com/kubernetes/kubernetes/pull/69279), [@dtaniwaki](https://github.com/dtaniwaki)) 1985 * add goroutine to move unschedulable pods to activeq if they are not retried for more than 1 minute ([#72558](https://github.com/kubernetes/kubernetes/pull/72558), [@denkensk](https://github.com/denkensk)) 1986 * PidPressure evicts pods from lowest priority to highest priority ([#72844](https://github.com/kubernetes/kubernetes/pull/72844), [@dashpole](https://github.com/dashpole)) 1987 * Reduce GCE log rotation check from 1 hour to every 5 minutes. Rotation policy is unchanged (new day starts, log file size > 100MB). ([#72062](https://github.com/kubernetes/kubernetes/pull/72062), [@jpbetz](https://github.com/jpbetz)) 1988 * Add support for max attach limit for Cinder ([#72980](https://github.com/kubernetes/kubernetes/pull/72980), [@gnufied](https://github.com/gnufied)) 1989 * Fixes the setting of NodeAddresses when using the vSphere CloudProvider and nodes that have multiple IP addresses. ([#70805](https://github.com/kubernetes/kubernetes/pull/70805), [@danwinship](https://github.com/danwinship)) 1990 * kubeadm: pull images when joining a new control plane instance ([#72870](https://github.com/kubernetes/kubernetes/pull/72870), [@MalloZup](https://github.com/MalloZup)) 1991 * Enable mTLS encryption between etcd and kube-apiserver in GCE ([#70144](https://github.com/kubernetes/kubernetes/pull/70144), [@wenjiaswe](https://github.com/wenjiaswe)) 1992 * The `/swaggerapi/*` schema docs, deprecated since 1.7, have been removed in favor of the /openapi/v2 schema docs. ([#72924](https://github.com/kubernetes/kubernetes/pull/72924), [@liggitt](https://github.com/liggitt)) 1993 * Allow users to use Docker 18.09 with kubeadm ([#72823](https://github.com/kubernetes/kubernetes/pull/72823), [@dims](https://github.com/dims)) 1994 1995 1996 1997 # v1.14.0-alpha.1 1998 1999 [Documentation](https://docs.k8s.io) 2000 2001 ## Downloads for v1.14.0-alpha.1 2002 2003 2004 filename | sha512 hash 2005 -------- | ----------- 2006 [kubernetes.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes.tar.gz) | `fac80e5674e547d00987516fb2eca6ea9947529307566be6a12932e3c9e430e8ad094afae748f31e9574838d98052423e3634a067f1456f7c13f6b27bfa63bcc` 2007 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-src.tar.gz) | `d1b5b2c15cb0daa076606f4ccf887724b0166dee0320f2a61d16ab4689931ab0cf5dac4c499aea3d434eb96d589d2b3effe0037e2244978d4290bd19b9a3edea` 2008 2009 ### Client Binaries 2010 2011 filename | sha512 hash 2012 -------- | ----------- 2013 [kubernetes-client-darwin-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-darwin-386.tar.gz) | `307c426e4abaf81648af393ddd641c225d87b02d8662d1309fe3528f14ed91b2470f6b46dc8ce0459cf196e2cec906f7eb972bf4c9a96cbd570e206f5a059dca` 2014 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-darwin-amd64.tar.gz) | `8daa85f3e8feaea0d55f20f850038dd113f0f08b62eef944b08a9109d4e69f323a8fcf20c12790c78386b454148bcc9a0cdf106ba3393620709d185c291887fa` 2015 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-linux-386.tar.gz) | `28d73c299cb9859fdfeb3e4869a7a9c77f5679309c2613bd2c72d92dafd5faad0653a7377616190edd29cb8fa1aff104daba98f398e72f3447a132f208dde756` 2016 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-linux-amd64.tar.gz) | `eb923e13026f80b743a57100d4f94995f322ab6f107c34ffd9aa74b5a6c6a4a410aff8921a4f675ace7db2ff8158a90874b8f56d3142ad2cbe615c11ec2d4535` 2017 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-linux-arm.tar.gz) | `279b0d0c560900021abea4bbfc25aeca7389f0b37d80022dc3335147344663424e7ba6a0abecb2dca1d2facb4163e26080750736a9a1932d67422f88b0940679` 2018 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-linux-arm64.tar.gz) | `d69d28361b9c9e16f3e6804ccda92d55ee743e63aba7fded04edf1f7202b1fa96c235e36ab2ca17df99b4aede80b92150790885bdb7f5b4d7956af3c269dd83c` 2019 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-linux-ppc64le.tar.gz) | `ca6ebb87df98bf179c94f54a4e8ae2ef2ea534b1bc5014331f937aa9d4c0442d5423651457871ef5c51f481ba8a3f449d69ef7e42e49c1b313f66cff3d44926f` 2020 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-linux-s390x.tar.gz) | `13fa2058ceba66d8da5ba5982aa302cdd1c61d15253183ab97739229584a178f057f7979b49a035cb2355197dbb388d1642939e2c002b10e23263127030022ab` 2021 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-windows-386.tar.gz) | `42ba4bba477e2958aab674a0fbf888bd5401fa5fbc39466b6cad0fc97e249ac949042c513bf176957bcb336a906e612d9c6790215e78c280225351236ec96993` 2022 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-client-windows-amd64.tar.gz) | `d5f339fe4d37c61babc97208446d1859423b7679f34040f72e9138b72a18d982e66732d1f4b4f3443700f9cbe96bfc0e12eaec0a8a373fb903b49efdafcbae04` 2023 2024 ### Server Binaries 2025 2026 filename | sha512 hash 2027 -------- | ----------- 2028 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-server-linux-amd64.tar.gz) | `bcbcbd3ac4419e54e894d1e595f883e61fcf9db0353a30d794a9e5030cde8957abe8124fa5265e8c52fbc93f07cfe79b2493f791dc225468bf927b7ab4694087` 2029 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-server-linux-arm.tar.gz) | `fda4ea9168555f724659601b06737dea6ec95574569df4ef7e4ab6c2cca3327623ef310bf34f792767f00ee8069b9dd83564835d43daf973087be816be40010b` 2030 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-server-linux-arm64.tar.gz) | `c142857711ec698844cd61188e70b5ab185ba2c8828cf5563a2f42958489e2ae4dbb2c1626271d4f5582167bb363e55ed03afb15e7e86cd414e0dc049fe384c0` 2031 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-server-linux-ppc64le.tar.gz) | `524a40c5717b24c5a3b2491c4c61cf3038ba5ae7f343797a1b56a5906d6a0a3eb57e9ae78590c28ac3d441d9d1bb480a0c264a07e009a4365503ad2357614aa8` 2032 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-server-linux-s390x.tar.gz) | `ef943fe326b05ece57f2e409ab1cc5fe863f5effa591abae17181c84a5eb4061e9f394ffcc8ee6ebb3f5165b183bab747a8cef540cbb1436343e8180cec037e0` 2033 2034 ### Node Binaries 2035 2036 filename | sha512 hash 2037 -------- | ----------- 2038 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-node-linux-amd64.tar.gz) | `396f7588e9131dd1b99d101c8bb94fb7e67ab067327ee58dab5a6e24887d8fbb6fc78fe50804abb0ab2f626034881d4280b3f678a1fd8b34891762bf2172b268` 2039 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-node-linux-arm.tar.gz) | `b75c1550438da0b66582d6de90436ee3c44e41e67f74947d93ee9a07ed2b7757762f3f2b05bd7b5589d7e1ea2eb3616b2ef4fe59a9fbe9d8e7cb8f0c9d3dd158` 2040 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-node-linux-arm64.tar.gz) | `b6c46f9250b5565fa178ecc99ffedc6724b0bfffb73acc7d3da2c678af71008a264502cc4a48a6e7452bd0a60d77194141bbc2ea9af49176ea66e27d874b77ac` 2041 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-node-linux-ppc64le.tar.gz) | `8d505c61a59bc9fc53d6f219d6434ddd962ba383654c46e16d413cee0ad6bd26f276a9860ad3680349bcfacb361e75de07fc44f7d14c054c47b6bd0eae63615f` 2042 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-node-linux-s390x.tar.gz) | `83b6cf0fb348faa93fa40ec2a947b202b3a5a2081c3896ae39618f947a57b431bc774fbe3a5437719f50f002de252438dc16bac6f632c11140f55d5051094ae6` 2043 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.14.0-alpha.1/kubernetes-node-windows-amd64.tar.gz) | `43471680533685c534023787cd40431b67041bab43e93dea457283ee0f08a8fa02ee9ade3737d8e64d1d3255a281af9a107cb61f9e4d9c99dee188c82a075580` 2044 2045 ## Changelog since v1.13.0 2046 2047 ### Action Required 2048 2049 * action required ([#68753](https://github.com/kubernetes/kubernetes/pull/68753), [@johnSchnake](https://github.com/johnSchnake)) 2050 * If you are running E2E tests which require SSH keys and you utilize environment variables to override their location, you may need to modify the environment variable set. On all providers the environment variable override can now be either an absolute path to the key or a relative path (relative to ~/.ssh). Specifically the changes are: 2051 * - Created new GCE_SSH_KEY allowing specification of SSH keys for gce, gke, and kubemark. 2052 * - AWS_SSH_KEY, previously assumed to be an absolute path can now be either relative or absolute 2053 * - LOCAL_SSH_KEY (for local and vsphere providers) was previously assumed to be a filename relative to ~/.ssh but can now also be an absolute path 2054 * - KUBE_SSH_KEY (for skeleton provider) was previously assumed to be a filename relative to ~/.ssh but can now also be an absolute path 2055 2056 ### Other notable changes 2057 2058 * Connections from Pods to Services with 0 endpoints will now ICMP reject immediately, rather than blackhole and timeout. ([#72534](https://github.com/kubernetes/kubernetes/pull/72534), [@thockin](https://github.com/thockin)) 2059 * Improve efficiency of preemption logic in clusters with many pending pods. ([#72895](https://github.com/kubernetes/kubernetes/pull/72895), [@bsalamat](https://github.com/bsalamat)) 2060 * Change scheduler metrics to conform metrics guidelines. ([#72332](https://github.com/kubernetes/kubernetes/pull/72332), [@danielqsj](https://github.com/danielqsj)) 2061 * The following metrics are deprecated, and will be removed in a future release: 2062 * `e2e_scheduling_latency_microseconds` 2063 * `scheduling_algorithm_latency_microseconds` 2064 * `scheduling_algorithm_predicate_evaluation` 2065 * `scheduling_algorithm_priority_evaluation` 2066 * `scheduling_algorithm_preemption_evaluation` 2067 * `binding_latency_microseconds` 2068 * Please convert to the following metrics: 2069 * `e2e_scheduling_latency_seconds` 2070 * `scheduling_algorithm_latency_seconds` 2071 * `scheduling_algorithm_predicate_evaluation_seconds` 2072 * `scheduling_algorithm_priority_evaluation_seconds` 2073 * `scheduling_algorithm_preemption_evaluation_seconds` 2074 * `binding_latency_seconds` 2075 * Fix SelectorSpreadPriority scheduler to match all selectors when distributing pods. ([#72801](https://github.com/kubernetes/kubernetes/pull/72801), [@Ramyak](https://github.com/Ramyak)) 2076 * Add bootstrap service account & cluster roles for node-lifecycle-controller, cloud-node-lifecycle-controller, and cloud-node-controller. ([#72764](https://github.com/kubernetes/kubernetes/pull/72764), [@andrewsykim](https://github.com/andrewsykim)) 2077 * Fixes spurious 0-length API responses. ([#72856](https://github.com/kubernetes/kubernetes/pull/72856), [@liggitt](https://github.com/liggitt)) 2078 * Updates Fluentd to 1.3.2 & added filter_parser ([#71180](https://github.com/kubernetes/kubernetes/pull/71180), [@monotek](https://github.com/monotek)) 2079 * The leaderelection package allows the lease holder to release its lease when the calling context is cancelled. This allows ([#71490](https://github.com/kubernetes/kubernetes/pull/71490), [@smarterclayton](https://github.com/smarterclayton)) 2080 * faster handoff when a leader-elected process is gracefully terminated. 2081 * Make volume binder resilient to races between main schedule loop and async binding operation ([#72045](https://github.com/kubernetes/kubernetes/pull/72045), [@cofyc](https://github.com/cofyc)) 2082 * Bump minimum docker API version to 1.26 (1.13.1) ([#72831](https://github.com/kubernetes/kubernetes/pull/72831), [@yujuhong](https://github.com/yujuhong)) 2083 * If the `TokenRequestProjection` feature gate is disabled, projected serviceAccountToken volume sources are now dropped at object creation time, or at object update time if the existing object did not have a projected serviceAccountToken volume source. Previously, these would result in validation errors. ([#72714](https://github.com/kubernetes/kubernetes/pull/72714), [@mourya007](https://github.com/mourya007)) 2084 * Add `metrics-port` to kube-proxy cmd flags. ([#72682](https://github.com/kubernetes/kubernetes/pull/72682), [@whypro](https://github.com/whypro)) 2085 * kubectl: fixed an issue with "too old resource version" errors continuously appearing when calling `kubectl delete` ([#72825](https://github.com/kubernetes/kubernetes/pull/72825), [@liggitt](https://github.com/liggitt)) 2086 * [Breaking change, client-go]: The WaitFor function returns, probably an ErrWaitTimeout, when the done channel is closed, even if the `WaitFunc` doesn't handle the done channel. ([#72364](https://github.com/kubernetes/kubernetes/pull/72364), [@kdada](https://github.com/kdada)) 2087 * removes newline from json output for windows nodes [#72657](https://github.com/kubernetes/kubernetes/pull/72657) ([#72659](https://github.com/kubernetes/kubernetes/pull/72659), [@jsturtevant](https://github.com/jsturtevant)) 2088 * The DenyEscalatingExec and DenyExecOnPrivileged admission plugins are deprecated and will be removed in v1.18. Use of `PodSecurityPolicy` or a custom admission plugin to limit creation of pods is recommended instead. ([#72737](https://github.com/kubernetes/kubernetes/pull/72737), [@liggitt](https://github.com/liggitt)) 2089 * Fix `describe statefulset` not printing number of desired replicas correctly ([#72781](https://github.com/kubernetes/kubernetes/pull/72781), [@tghartland](https://github.com/tghartland)) 2090 * Fix kube-proxy PodSecurityPolicy binding on GCE & GKE. This was only an issue when running kube-proxy as a DaemonSet, with PodSecurityPolicy enabled. ([#72761](https://github.com/kubernetes/kubernetes/pull/72761), [@tallclair](https://github.com/tallclair)) 2091 * Drops `status.Conditions` of new `PersistentVolume` objects if it was not set on the old object during `PrepareForUpdate`. ([#72739](https://github.com/kubernetes/kubernetes/pull/72739), [@rajathagasthya](https://github.com/rajathagasthya)) 2092 * kubelet: fixes cadvisor internal error when "--container-runtime-endpoint" is set to "unix:///var/run/crio/crio.sock". ([#72340](https://github.com/kubernetes/kubernetes/pull/72340), [@makocchi-git](https://github.com/makocchi-git)) 2093 * The `spec.SecurityContext.Sysctls` field is now dropped during creation of `Pod` objects unless the `Sysctls` feature gate is enabled. ([#72752](https://github.com/kubernetes/kubernetes/pull/72752), [@rajathagasthya](https://github.com/rajathagasthya)) 2094 * The `spec.AllowedUnsafeSysctls` and `spec.ForbiddenSysctls` fields are now dropped during creation of `PodSecurityPolicy` objects unless the `Sysctls` feature gate is enabled. 2095 * kubeadm: fixed storing of front-proxy certificate in secrets required by kube-controller-manager selfhosting pivoting ([#72727](https://github.com/kubernetes/kubernetes/pull/72727), [@bart0sh](https://github.com/bart0sh)) 2096 * Administrator is able to configure max pids for a pod on a node. ([#72076](https://github.com/kubernetes/kubernetes/pull/72076), [@derekwaynecarr](https://github.com/derekwaynecarr)) 2097 * Move users of `factory.NewConfigFactory` to `scheduler.New`. ([#71875](https://github.com/kubernetes/kubernetes/pull/71875), [@wgliang](https://github.com/wgliang)) 2098 * The `spec.SecurityContext.ShareProcessNamespace` field is now dropped during creation of `Pod` objects unless the `PodShareProcessNamespace ` feature gate is enabled. ([#72698](https://github.com/kubernetes/kubernetes/pull/72698), [@rajathagasthya](https://github.com/rajathagasthya)) 2099 * kube-apiserver: When configuring integration with external KMS Providers, users can supply timeout value (i.e. how long should kube-apiserver wait before giving up on a call to KMS). ([#72540](https://github.com/kubernetes/kubernetes/pull/72540), [@immutableT](https://github.com/immutableT)) 2100 * The `spec.readinessGates` field is now dropped during creation of `Pod` objects unless the `PodReadinessGates` feature gate is enabled. ([#72695](https://github.com/kubernetes/kubernetes/pull/72695), [@rajathagasthya](https://github.com/rajathagasthya)) 2101 * The `spec.dataSource` field is now dropped during creation of PersistentVolumeClaim objects unless the `VolumeSnapshotDataSource` feature gate is enabled. ([#72666](https://github.com/kubernetes/kubernetes/pull/72666), [@rajathagasthya](https://github.com/rajathagasthya)) 2102 * Stop kubelet logging a warning to override hostname if there's no change detected. ([#71560](https://github.com/kubernetes/kubernetes/pull/71560), [@KashifSaadat](https://github.com/KashifSaadat)) 2103 * client-go: fake clients now properly return NotFound errors when attempting to patch non-existent objects ([#70886](https://github.com/kubernetes/kubernetes/pull/70886), [@bouk](https://github.com/bouk)) 2104 * kubectl: fixes a bug determining the correct namespace while running in a pod when the `--context` flag is explicitly specified, and the referenced context specifies the namespace `default` ([#72529](https://github.com/kubernetes/kubernetes/pull/72529), [@liggitt](https://github.com/liggitt)) 2105 * Fix scheduling starvation of pods in cluster with large number of unschedulable pods. ([#72619](https://github.com/kubernetes/kubernetes/pull/72619), [@everpeace](https://github.com/everpeace)) 2106 * If the AppArmor feature gate is disabled, AppArmor-specific annotations in pod and pod templates are dropped when the object is created, and during update of objects that do not already contain AppArmor annotations, rather than triggering a validation error. ([#72655](https://github.com/kubernetes/kubernetes/pull/72655), [@liggitt](https://github.com/liggitt)) 2107 * client-go: shortens refresh period for token files to 1 minute to ensure auto-rotated projected service account tokens are read frequently enough. ([#72437](https://github.com/kubernetes/kubernetes/pull/72437), [@liggitt](https://github.com/liggitt)) 2108 * Multiple tests which previously failed due to lack of external IP addresses defined on the nodes should now be passable. ([#68792](https://github.com/kubernetes/kubernetes/pull/68792), [@johnSchnake](https://github.com/johnSchnake)) 2109 * kubeadm: fixed incorrect controller manager pod mutations during selfhosting pivoting ([#72518](https://github.com/kubernetes/kubernetes/pull/72518), [@bart0sh](https://github.com/bart0sh)) 2110 * Increase Azure default maximumLoadBalancerRuleCount to 250. ([#72621](https://github.com/kubernetes/kubernetes/pull/72621), [@feiskyer](https://github.com/feiskyer)) 2111 * RuntimeClass is now printed with extra `RUNTIME-HANDLER` column. ([#72446](https://github.com/kubernetes/kubernetes/pull/72446), [@Huang-Wei](https://github.com/Huang-Wei)) 2112 * Updates the kubernetes dashboard add-on to v1.10.1. Skipping dashboard login is no longer enabled by default. ([#72495](https://github.com/kubernetes/kubernetes/pull/72495), [@liggitt](https://github.com/liggitt)) 2113 * [GCP] Remove confusing error log entry form fluentd scalers. ([#72243](https://github.com/kubernetes/kubernetes/pull/72243), [@cezarygerard](https://github.com/cezarygerard)) 2114 * change azure disk host cache to ReadOnly by default ([#72229](https://github.com/kubernetes/kubernetes/pull/72229), [@andyzhangx](https://github.com/andyzhangx)) 2115 * Nodes deleted in the cloud provider with Ready condition `Unknown` should also be deleted on the API server. ([#72559](https://github.com/kubernetes/kubernetes/pull/72559), [@andrewsykim](https://github.com/andrewsykim)) 2116 * `kubectl apply --prune` now uses the apps/v1 API to prune workload resources ([#72352](https://github.com/kubernetes/kubernetes/pull/72352), [@liggitt](https://github.com/liggitt)) 2117 * Fixes a bug in HPA controller so HPAs are always updated every resyncPeriod (15 seconds). ([#72373](https://github.com/kubernetes/kubernetes/pull/72373), [@krzysztof-jastrzebski](https://github.com/krzysztof-jastrzebski)) 2118 * IPVS: "ExternalTrafficPolicy: Local" now works with LoadBalancer services using loadBalancerIP ([#72432](https://github.com/kubernetes/kubernetes/pull/72432), [@lbernail](https://github.com/lbernail)) 2119 * Fixes issue with cleaning up stale NFS subpath mounts ([#71804](https://github.com/kubernetes/kubernetes/pull/71804), [@msau42](https://github.com/msau42)) 2120 * Modify the scheduling result struct and improve logging for successful binding. ([#71926](https://github.com/kubernetes/kubernetes/pull/71926), [@wgliang](https://github.com/wgliang)) 2121 * Run one etcd storage compaction per default interval of 5min. Do not run one for each resource and each CRD. This fixes the compaction log spam and reduces load on etcd. ([#68557](https://github.com/kubernetes/kubernetes/pull/68557), [@sttts](https://github.com/sttts)) 2122 * kube-apiserver: `--runtime-config` can once again be used to enable/disable serving specific resources in the `extensions/v1beta1` API group. Note that specific resource enablement/disablement is only allowed for the `extensions/v1beta1` API group for legacy reasons. Attempts to enable/disable individual resources in other API groups will print a warning, and will return an error in future releases. ([#72249](https://github.com/kubernetes/kubernetes/pull/72249), [@liggitt](https://github.com/liggitt)) 2123 * kubeadm: fixed storing of etcd certificates in secrets required by kube-apiserver selfhosting pivoting ([#72478](https://github.com/kubernetes/kubernetes/pull/72478), [@bart0sh](https://github.com/bart0sh)) 2124 * kubeadm: remove the deprecated "--address" flag for controller-manager and scheduler. ([#71973](https://github.com/kubernetes/kubernetes/pull/71973), [@MalloZup](https://github.com/MalloZup)) 2125 * kube-apiserver: improves performance of requests made with service account token authentication ([#71816](https://github.com/kubernetes/kubernetes/pull/71816), [@liggitt](https://github.com/liggitt)) 2126 * Use prometheus conventions for workqueue metrics. ([#71300](https://github.com/kubernetes/kubernetes/pull/71300), [@danielqsj](https://github.com/danielqsj)) 2127 * It is now deprecated to use the following metrics: 2128 * `{WorkQueueName}_depth` 2129 * `{WorkQueueName}_adds` 2130 * `{WorkQueueName}_queue_latency` 2131 * `{WorkQueueName}_work_duration` 2132 * `{WorkQueueName}_unfinished_work_seconds` 2133 * `{WorkQueueName}_longest_running_processor_microseconds` 2134 * `{WorkQueueName}_retries` 2135 * Please convert to the following metrics: 2136 * `workqueue_depth` 2137 * `workqueue_adds_total` 2138 * `workqueue_queue_latency_seconds` 2139 * `workqueue_work_duration_seconds` 2140 * `workqueue_unfinished_work_seconds` 2141 * `workqueue_longest_running_processor_seconds` 2142 * `workqueue_retries_total` 2143 * Fix inability to use k8s with dockerd having default IPC mode set to private. ([#70826](https://github.com/kubernetes/kubernetes/pull/70826), [@kolyshkin](https://github.com/kolyshkin)) 2144 * Fix a race condition in the scheduler preemption logic that could cause nominatedNodeName of a pod not to be considered in one or more scheduling cycles. ([#72259](https://github.com/kubernetes/kubernetes/pull/72259), [@bsalamat](https://github.com/bsalamat)) 2145 * Fix registration for scheduling framework plugins with the default plugin set ([#72396](https://github.com/kubernetes/kubernetes/pull/72396), [@y-taka-23](https://github.com/y-taka-23)) 2146 * The GA VolumeScheduling feature gate can no longer be disabled and will be removed in a future release ([#72382](https://github.com/kubernetes/kubernetes/pull/72382), [@liggitt](https://github.com/liggitt)) 2147 * Fix race condition introduced by graceful termination which can lead to a deadlock in kube-proxy ([#72361](https://github.com/kubernetes/kubernetes/pull/72361), [@lbernail](https://github.com/lbernail)) 2148 * Fixes issue where subpath volume content was deleted during orphaned pod cleanup for Local volumes that are directories (and not mount points) on the root filesystem. ([#72291](https://github.com/kubernetes/kubernetes/pull/72291), [@msau42](https://github.com/msau42)) 2149 * Fixes `kubectl create secret docker-registry` compatibility ([#72344](https://github.com/kubernetes/kubernetes/pull/72344), [@liggitt](https://github.com/liggitt)) 2150 * Add-on manifests now use the apps/v1 API for DaemonSets, Deployments, and ReplicaSets ([#72203](https://github.com/kubernetes/kubernetes/pull/72203), [@liggitt](https://github.com/liggitt)) 2151 * "kubectl wait" command now supports the "--all" flag to select all resources in the namespace of the specified resource types. ([#70599](https://github.com/kubernetes/kubernetes/pull/70599), [@caesarxuchao](https://github.com/caesarxuchao)) 2152 * `deployments/rollback` is now passed through validation/admission controllers ([#72271](https://github.com/kubernetes/kubernetes/pull/72271), [@jhrv](https://github.com/jhrv)) 2153 * The `Lease` API type in the `coordination.k8s.io` API group is promoted to `v1` ([#72239](https://github.com/kubernetes/kubernetes/pull/72239), [@wojtek-t](https://github.com/wojtek-t)) 2154 * Move compatibility_test.go to pkg/scheduler/api ([#72014](https://github.com/kubernetes/kubernetes/pull/72014), [@huynq0911](https://github.com/huynq0911)) 2155 * New Azure cloud provider option 'cloudProviderBackoffMode' has been added to reduce Azure API retries. Candidate values are: ([#70866](https://github.com/kubernetes/kubernetes/pull/70866), [@feiskyer](https://github.com/feiskyer)) 2156 * default (or empty string): keep same with before. 2157 * v2: only backoff retry with Azure SDK with fixed exponent 2. 2158 * Set percentage of nodes scored in each cycle dynamically based on the cluster size. ([#72140](https://github.com/kubernetes/kubernetes/pull/72140), [@wgliang](https://github.com/wgliang)) 2159 * Fix AAD support for Azure sovereign cloud in kubectl ([#72143](https://github.com/kubernetes/kubernetes/pull/72143), [@karataliu](https://github.com/karataliu)) 2160 * Make kube-proxy service abstraction optional. ([#71355](https://github.com/kubernetes/kubernetes/pull/71355), [@bradhoekstra](https://github.com/bradhoekstra)) 2161 * Add the 'service.kubernetes.io/service-proxy-name' label to a Service to disable the kube-proxy service proxy implementation. 2162 * kubectl: `-A` can now be used as a shortcut for `--all-namespaces` ([#72006](https://github.com/kubernetes/kubernetes/pull/72006), [@soltysh](https://github.com/soltysh)) 2163 * discovery.CachedDiscoveryInterface implementation returned by NewMemCacheClient has changed semantics of Invalidate method -- the cache refresh is now deferred to the first cache lookup. ([#70994](https://github.com/kubernetes/kubernetes/pull/70994), [@mborsz](https://github.com/mborsz)) 2164 * Fix device mountable volume names in DSW to prevent races in device mountable plugin, e.g. local. ([#71509](https://github.com/kubernetes/kubernetes/pull/71509), [@cofyc](https://github.com/cofyc)) 2165 * Enable customize in kubectl: kubectl will be able to recognize directories with kustomization.YAML ([#70875](https://github.com/kubernetes/kubernetes/pull/70875), [@Liujingfang1](https://github.com/Liujingfang1)) 2166 * Stably sort controllerrevisions. This can prevent pods of statefulsets from continually rolling. ([#66882](https://github.com/kubernetes/kubernetes/pull/66882), [@ryanmcnamara](https://github.com/ryanmcnamara)) 2167 * Update to use go1.11.4. ([#72084](https://github.com/kubernetes/kubernetes/pull/72084), [@ixdy](https://github.com/ixdy)) 2168 * fixes an issue deleting pods containing subpath volume mounts with the VolumeSubpath feature disabled ([#70490](https://github.com/kubernetes/kubernetes/pull/70490), [@liggitt](https://github.com/liggitt)) 2169 * Clean up old eclass code ([#71399](https://github.com/kubernetes/kubernetes/pull/71399), [@resouer](https://github.com/resouer)) 2170 * Fix a race condition in which kubeadm only waits for the kubelets kubeconfig file when it has performed the TLS bootstrap, but wasn't waiting for certificates to be present in the filesystem ([#72030](https://github.com/kubernetes/kubernetes/pull/72030), [@ereslibre](https://github.com/ereslibre)) 2171 * In addition to restricting GCE metadata requests to known APIs, the metadata-proxy now restricts query strings to known parameters. ([#71094](https://github.com/kubernetes/kubernetes/pull/71094), [@dekkagaijin](https://github.com/dekkagaijin)) 2172 * kubeadm: fix a possible panic when joining a new control plane node in HA scenarios ([#72123](https://github.com/kubernetes/kubernetes/pull/72123), [@anitgandhi](https://github.com/anitgandhi)) 2173 * fix race condition when attach azure disk in vmss ([#71992](https://github.com/kubernetes/kubernetes/pull/71992), [@andyzhangx](https://github.com/andyzhangx)) 2174 * Update to use go1.11.3 with fix for CVE-2018-16875 ([#72035](https://github.com/kubernetes/kubernetes/pull/72035), [@seemethere](https://github.com/seemethere)) 2175 * kubeadm: fix a bug when syncing etcd endpoints ([#71945](https://github.com/kubernetes/kubernetes/pull/71945), [@pytimer](https://github.com/pytimer)) 2176 * fix kubelet log flushing issue in azure disk ([#71990](https://github.com/kubernetes/kubernetes/pull/71990), [@andyzhangx](https://github.com/andyzhangx)) 2177 * Disable proxy to loopback and linklocal ([#71980](https://github.com/kubernetes/kubernetes/pull/71980), [@micahhausler](https://github.com/micahhausler)) 2178 * Fix overlapping filenames in diff if multiple resources have the same name. ([#71923](https://github.com/kubernetes/kubernetes/pull/71923), [@apelisse](https://github.com/apelisse)) 2179 * fix issue: vm sku restriction policy does not work in azure disk attach/detach ([#71941](https://github.com/kubernetes/kubernetes/pull/71941), [@andyzhangx](https://github.com/andyzhangx)) 2180 * kubeadm: Create /var/lib/etcd with correct permissions (0700) by default. ([#71885](https://github.com/kubernetes/kubernetes/pull/71885), [@dims](https://github.com/dims)) 2181 * Scheduler only activates unschedulable pods if node's scheduling related properties change. ([#71551](https://github.com/kubernetes/kubernetes/pull/71551), [@mlmhl](https://github.com/mlmhl)) 2182 * kube-proxy in IPVS mode will stop initiating connections to terminating pods for services with sessionAffinity set. ([#71834](https://github.com/kubernetes/kubernetes/pull/71834), [@lbernail](https://github.com/lbernail)) 2183 * kubeadm: improve hostport parsing error messages ([#71258](https://github.com/kubernetes/kubernetes/pull/71258), [@bart0sh](https://github.com/bart0sh)) 2184 * Support graceful termination with IPVS when deleting a service ([#71895](https://github.com/kubernetes/kubernetes/pull/71895), [@lbernail](https://github.com/lbernail)) 2185 * Include CRD for BGPConfigurations, needed for calico 2.x to 3.x upgrade. ([#71868](https://github.com/kubernetes/kubernetes/pull/71868), [@satyasm](https://github.com/satyasm)) 2186 * apply: fix detection of non-dry-run enabled servers ([#71854](https://github.com/kubernetes/kubernetes/pull/71854), [@apelisse](https://github.com/apelisse)) 2187 * Clear UDP conntrack entry on endpoint changes when using nodeport ([#71573](https://github.com/kubernetes/kubernetes/pull/71573), [@JacobTanenbaum](https://github.com/JacobTanenbaum)) 2188 * Add successful and failed history limits to cronjob describe ([#71844](https://github.com/kubernetes/kubernetes/pull/71844), [@soltysh](https://github.com/soltysh)) 2189 * kube-controller-manager: fixed issue display help for the deprecated insecure --port flag ([#71601](https://github.com/kubernetes/kubernetes/pull/71601), [@liggitt](https://github.com/liggitt)) 2190 * kubectl: fixes regression in --sort-by behavior ([#71805](https://github.com/kubernetes/kubernetes/pull/71805), [@liggitt](https://github.com/liggitt)) 2191 * Fixes pod deletion when cleaning old cronjobs ([#71801](https://github.com/kubernetes/kubernetes/pull/71801), [@soltysh](https://github.com/soltysh)) 2192 * kubeadm: use kubeconfig flag instead of kubeconfig-dir on init phase bootstrap-token ([#71803](https://github.com/kubernetes/kubernetes/pull/71803), [@yagonobre](https://github.com/yagonobre)) 2193 * kube-scheduler: restores ability to run without authentication configuration lookup permissions ([#71755](https://github.com/kubernetes/kubernetes/pull/71755), [@liggitt](https://github.com/liggitt)) 2194 * Add aggregator_unavailable_apiservice_{count,gauge} metrics in the kube-aggregator. ([#71380](https://github.com/kubernetes/kubernetes/pull/71380), [@sttts](https://github.com/sttts)) 2195 * Fixes apiserver nil pointer panics when requesting v2beta1 autoscaling object metrics ([#71744](https://github.com/kubernetes/kubernetes/pull/71744), [@yue9944882](https://github.com/yue9944882)) 2196 * Only use the first IP address got from instance metadata. This is because Azure CNI would set up a list of IP addresses in instance metadata, while only the first one is the Node's IP. ([#71736](https://github.com/kubernetes/kubernetes/pull/71736), [@feiskyer](https://github.com/feiskyer)) 2197 * client-go: restores behavior of populating the BearerToken field in rest.Config objects constructed from kubeconfig files containing tokenFile config, or from in-cluster configuration. An additional BearerTokenFile field is now populated to enable constructed clients to periodically refresh tokens. ([#71713](https://github.com/kubernetes/kubernetes/pull/71713), [@liggitt](https://github.com/liggitt)) 2198 * kubeadm: remove deprecated kubeadm config print-defaults command ([#71467](https://github.com/kubernetes/kubernetes/pull/71467), [@rosti](https://github.com/rosti)) 2199 * hack/local-up-cluster.sh now enables kubelet authentication/authorization by default (they can be disabled with KUBELET_AUTHENTICATION_WEBHOOK=false and KUBELET_AUTHORIZATION_WEBHOOK=false ([#71690](https://github.com/kubernetes/kubernetes/pull/71690), [@liggitt](https://github.com/liggitt)) 2200 * Fixes an issue where Azure VMSS instances not existing in Azure were not being deleted by the Cloud Controller Manager. ([#71597](https://github.com/kubernetes/kubernetes/pull/71597), [@marc-sensenich](https://github.com/marc-sensenich)) 2201 * kubeadm reset correctly unmounts mount points inside /var/lib/kubelet ([#71663](https://github.com/kubernetes/kubernetes/pull/71663), [@bart0sh](https://github.com/bart0sh)) 2202 * Upgrade default etcd server to 3.3.10 ([#71615](https://github.com/kubernetes/kubernetes/pull/71615), [@jpbetz](https://github.com/jpbetz)) 2203 * When creating a service with annotation: service.beta.kubernetes.io/load-balancer-source-ranges containing multiple source ranges and service.beta.kubernetes.io/azure-shared-securityrule: "false", the NSG rules will be collapsed. ([#71484](https://github.com/kubernetes/kubernetes/pull/71484), [@ritazh](https://github.com/ritazh)) 2204 * disable node's proxy use of http probe ([#68663](https://github.com/kubernetes/kubernetes/pull/68663), [@WanLinghao](https://github.com/WanLinghao)) 2205 * Bumps version of kubernetes-cni to 0.6.0 ([#71629](https://github.com/kubernetes/kubernetes/pull/71629), [@mauilion](https://github.com/mauilion)) 2206 * On GCI, NPD starts to monitor kubelet, docker, containerd crashlooping, read-only filesystem and corrupt docker overlay2 issues. ([#71522](https://github.com/kubernetes/kubernetes/pull/71522), [@wangzhen127](https://github.com/wangzhen127)) 2207 * When a kubelet is using --bootstrap-kubeconfig and certificate rotation, it no longer waits for bootstrap to succeed before launching static pods. ([#71174](https://github.com/kubernetes/kubernetes/pull/71174), [@smarterclayton](https://github.com/smarterclayton)) 2208 * Add an plugin interfaces for "reserve" and "prebind" extension points of the scheduling framework. ([#70227](https://github.com/kubernetes/kubernetes/pull/70227), [@bsalamat](https://github.com/bsalamat)) 2209 * Fix scheduling starvation of pods in cluster with large number of unschedulable pods. ([#71488](https://github.com/kubernetes/kubernetes/pull/71488), [@bsalamat](https://github.com/bsalamat)) 2210 * Reduce CSI log and event spam. ([#71581](https://github.com/kubernetes/kubernetes/pull/71581), [@saad-ali](https://github.com/saad-ali)) 2211 * Add conntrack as a dependency of kubelet and kubeadm when building rpms and debs. Both require conntrack to handle cleanup of connections. ([#71540](https://github.com/kubernetes/kubernetes/pull/71540), [@mauilion](https://github.com/mauilion)) 2212 * UDP connections now support graceful termination in IPVS mode ([#71515](https://github.com/kubernetes/kubernetes/pull/71515), [@lbernail](https://github.com/lbernail)) 2213 * Log etcd client errors. The verbosity is set with the usual `-v` flag. ([#71318](https://github.com/kubernetes/kubernetes/pull/71318), [@sttts](https://github.com/sttts)) 2214 * The `DefaultFeatureGate` package variable now only exposes readonly feature gate methods. Methods for mutating feature gates have moved into a `MutableFeatureGate` interface and are accessible via the `DefaultMutableFeatureGate` package variable. Only top-level commands and options setup should access `DefaultMutableFeatureGate`. ([#71302](https://github.com/kubernetes/kubernetes/pull/71302), [@liggitt](https://github.com/liggitt)) 2215 * `node.kubernetes.io/pid-pressure` toleration is added for DaemonSet pods, and `node.kubernetes.io/out-of-disk` isn't added any more even if it's a critical pod. ([#67036](https://github.com/kubernetes/kubernetes/pull/67036), [@Huang-Wei](https://github.com/Huang-Wei)) 2216 * Update k8s.io/utils to allow for asynchronous process control ([#71047](https://github.com/kubernetes/kubernetes/pull/71047), [@hoegaarden](https://github.com/hoegaarden)) 2217 * Fixes possible panic during volume detach, if corresponding volume plugin became non-attachable ([#71471](https://github.com/kubernetes/kubernetes/pull/71471), [@mshaverdo](https://github.com/mshaverdo)) 2218 * Fix cloud-controller-manager crash when using AWS provider and PersistentVolume initializing controller ([#70432](https://github.com/kubernetes/kubernetes/pull/70432), [@mvladev](https://github.com/mvladev)) 2219 * Fixes an issue where Portworx volumes cannot be mounted if 9001 port is already in use on the host and users remap 9001 to another port. ([#70392](https://github.com/kubernetes/kubernetes/pull/70392), [@harsh-px](https://github.com/harsh-px)) 2220 * Fix `SubPath` printing of `VolumeMounts`. ([#70127](https://github.com/kubernetes/kubernetes/pull/70127), [@dtaniwaki](https://github.com/dtaniwaki)) 2221 * Fixes incorrect paths (missing first letter) when copying files from pods to ([#69885](https://github.com/kubernetes/kubernetes/pull/69885), [@clickyotomy](https://github.com/clickyotomy)) 2222 * local in `kubectl cp'. 2223 * Fix AWS NLB security group updates where valid security group ports were incorrectly removed ([#68422](https://github.com/kubernetes/kubernetes/pull/68422), [@kellycampbell](https://github.com/kellycampbell)) 2224 * when updating a service or when node changes occur. 2225