k8s.io/kubernetes@v1.31.0-alpha.0.0.20240520171757-56147500dadc/CHANGELOG/CHANGELOG-1.23.md (about) 1 <!-- BEGIN MUNGE: GENERATED_TOC --> 2 3 - [v1.23.17](#v12317) 4 - [Downloads for v1.23.17](#downloads-for-v12317) 5 - [Source Code](#source-code) 6 - [Client Binaries](#client-binaries) 7 - [Server Binaries](#server-binaries) 8 - [Node Binaries](#node-binaries) 9 - [Container Images](#container-images) 10 - [Changelog since v1.23.16](#changelog-since-v12316) 11 - [Changes by Kind](#changes-by-kind) 12 - [Feature](#feature) 13 - [Bug or Regression](#bug-or-regression) 14 - [Dependencies](#dependencies) 15 - [Added](#added) 16 - [Changed](#changed) 17 - [Removed](#removed) 18 - [v1.23.16](#v12316) 19 - [Downloads for v1.23.16](#downloads-for-v12316) 20 - [Source Code](#source-code-1) 21 - [Client Binaries](#client-binaries-1) 22 - [Server Binaries](#server-binaries-1) 23 - [Node Binaries](#node-binaries-1) 24 - [Container Images](#container-images-1) 25 - [Changelog since v1.23.15](#changelog-since-v12315) 26 - [Changes by Kind](#changes-by-kind-1) 27 - [API Change](#api-change) 28 - [Feature](#feature-1) 29 - [Bug or Regression](#bug-or-regression-1) 30 - [Dependencies](#dependencies-1) 31 - [Added](#added-1) 32 - [Changed](#changed-1) 33 - [Removed](#removed-1) 34 - [v1.23.15](#v12315) 35 - [Downloads for v1.23.15](#downloads-for-v12315) 36 - [Source Code](#source-code-2) 37 - [Client Binaries](#client-binaries-2) 38 - [Server Binaries](#server-binaries-2) 39 - [Node Binaries](#node-binaries-2) 40 - [Container Images](#container-images-2) 41 - [Changelog since v1.23.14](#changelog-since-v12314) 42 - [Changes by Kind](#changes-by-kind-2) 43 - [Feature](#feature-2) 44 - [Bug or Regression](#bug-or-regression-2) 45 - [Other (Cleanup or Flake)](#other-cleanup-or-flake) 46 - [Dependencies](#dependencies-2) 47 - [Added](#added-2) 48 - [Changed](#changed-2) 49 - [Removed](#removed-2) 50 - [v1.23.14](#v12314) 51 - [Downloads for v1.23.14](#downloads-for-v12314) 52 - [Source Code](#source-code-3) 53 - [Client Binaries](#client-binaries-3) 54 - [Server Binaries](#server-binaries-3) 55 - [Node Binaries](#node-binaries-3) 56 - [Container Images](#container-images-3) 57 - [Changelog since v1.23.13](#changelog-since-v12313) 58 - [Important Security Information](#important-security-information) 59 - [CVE-2022-3162: Unauthorized read of Custom Resources](#cve-2022-3162-unauthorized-read-of-custom-resources) 60 - [CVE-2022-3294: Node address isn't always verified when proxying](#cve-2022-3294-node-address-isnt-always-verified-when-proxying) 61 - [Changes by Kind](#changes-by-kind-3) 62 - [API Change](#api-change-1) 63 - [Bug or Regression](#bug-or-regression-3) 64 - [Dependencies](#dependencies-3) 65 - [Added](#added-3) 66 - [Changed](#changed-3) 67 - [Removed](#removed-3) 68 - [v1.23.13](#v12313) 69 - [Downloads for v1.23.13](#downloads-for-v12313) 70 - [Source Code](#source-code-4) 71 - [Client Binaries](#client-binaries-4) 72 - [Server Binaries](#server-binaries-4) 73 - [Node Binaries](#node-binaries-4) 74 - [Container Images](#container-images-4) 75 - [Changelog since v1.23.12](#changelog-since-v12312) 76 - [Changes by Kind](#changes-by-kind-4) 77 - [Bug or Regression](#bug-or-regression-4) 78 - [Dependencies](#dependencies-4) 79 - [Added](#added-4) 80 - [Changed](#changed-4) 81 - [Removed](#removed-4) 82 - [v1.23.12](#v12312) 83 - [Downloads for v1.23.12](#downloads-for-v12312) 84 - [Source Code](#source-code-5) 85 - [Client Binaries](#client-binaries-5) 86 - [Server Binaries](#server-binaries-5) 87 - [Node Binaries](#node-binaries-5) 88 - [Container Images](#container-images-5) 89 - [Changelog since v1.23.11](#changelog-since-v12311) 90 - [Changes by Kind](#changes-by-kind-5) 91 - [Bug or Regression](#bug-or-regression-5) 92 - [Dependencies](#dependencies-5) 93 - [Added](#added-5) 94 - [Changed](#changed-5) 95 - [Removed](#removed-5) 96 - [v1.23.11](#v12311) 97 - [Downloads for v1.23.11](#downloads-for-v12311) 98 - [Source Code](#source-code-6) 99 - [Client Binaries](#client-binaries-6) 100 - [Server Binaries](#server-binaries-6) 101 - [Node Binaries](#node-binaries-6) 102 - [Container Images](#container-images-6) 103 - [Changelog since v1.23.10](#changelog-since-v12310) 104 - [Important Security Information](#important-security-information-1) 105 - [CVE-2022-3172: Aggregated API server can cause clients to be redirected (SSRF)](#cve-2022-3172-aggregated-api-server-can-cause-clients-to-be-redirected-ssrf) 106 - [CVE-2021-25749: <code>runAsNonRoot</code> logic bypass for Windows containers](#cve-2021-25749-runasnonroot-logic-bypass-for-windows-containers) 107 - [Changes by Kind](#changes-by-kind-6) 108 - [Bug or Regression](#bug-or-regression-6) 109 - [Dependencies](#dependencies-6) 110 - [Added](#added-6) 111 - [Changed](#changed-6) 112 - [Removed](#removed-6) 113 - [v1.23.10](#v12310) 114 - [Downloads for v1.23.10](#downloads-for-v12310) 115 - [Source Code](#source-code-7) 116 - [Client Binaries](#client-binaries-7) 117 - [Server Binaries](#server-binaries-7) 118 - [Node Binaries](#node-binaries-7) 119 - [Container Images](#container-images-7) 120 - [Changelog since v1.23.9](#changelog-since-v1239) 121 - [Changes by Kind](#changes-by-kind-7) 122 - [Feature](#feature-3) 123 - [Bug or Regression](#bug-or-regression-7) 124 - [Dependencies](#dependencies-7) 125 - [Added](#added-7) 126 - [Changed](#changed-7) 127 - [Removed](#removed-7) 128 - [v1.23.9](#v1239) 129 - [Downloads for v1.23.9](#downloads-for-v1239) 130 - [Source Code](#source-code-8) 131 - [Client Binaries](#client-binaries-8) 132 - [Server Binaries](#server-binaries-8) 133 - [Node Binaries](#node-binaries-8) 134 - [Container Images](#container-images-8) 135 - [Changelog since v1.23.8](#changelog-since-v1238) 136 - [Changes by Kind](#changes-by-kind-8) 137 - [Bug or Regression](#bug-or-regression-8) 138 - [Dependencies](#dependencies-8) 139 - [Added](#added-8) 140 - [Changed](#changed-8) 141 - [Removed](#removed-8) 142 - [v1.23.8](#v1238) 143 - [Downloads for v1.23.8](#downloads-for-v1238) 144 - [Source Code](#source-code-9) 145 - [Client Binaries](#client-binaries-9) 146 - [Server Binaries](#server-binaries-9) 147 - [Node Binaries](#node-binaries-9) 148 - [Container Images](#container-images-9) 149 - [Changelog since v1.23.7](#changelog-since-v1237) 150 - [Changes by Kind](#changes-by-kind-9) 151 - [Feature](#feature-4) 152 - [Bug or Regression](#bug-or-regression-9) 153 - [Dependencies](#dependencies-9) 154 - [Added](#added-9) 155 - [Changed](#changed-9) 156 - [Removed](#removed-9) 157 - [v1.23.7](#v1237) 158 - [Downloads for v1.23.7](#downloads-for-v1237) 159 - [Source Code](#source-code-10) 160 - [Client Binaries](#client-binaries-10) 161 - [Server Binaries](#server-binaries-10) 162 - [Node Binaries](#node-binaries-10) 163 - [Container Images](#container-images-10) 164 - [Changelog since v1.23.6](#changelog-since-v1236) 165 - [Changes by Kind](#changes-by-kind-10) 166 - [API Change](#api-change-2) 167 - [Feature](#feature-5) 168 - [Failing Test](#failing-test) 169 - [Bug or Regression](#bug-or-regression-10) 170 - [Dependencies](#dependencies-10) 171 - [Added](#added-10) 172 - [Changed](#changed-10) 173 - [Removed](#removed-10) 174 - [v1.23.6](#v1236) 175 - [Downloads for v1.23.6](#downloads-for-v1236) 176 - [Source Code](#source-code-11) 177 - [Client Binaries](#client-binaries-11) 178 - [Server Binaries](#server-binaries-11) 179 - [Node Binaries](#node-binaries-11) 180 - [Container Images](#container-images-11) 181 - [Changelog since v1.23.5](#changelog-since-v1235) 182 - [Changes by Kind](#changes-by-kind-11) 183 - [API Change](#api-change-3) 184 - [Feature](#feature-6) 185 - [Bug or Regression](#bug-or-regression-11) 186 - [Dependencies](#dependencies-11) 187 - [Added](#added-11) 188 - [Changed](#changed-11) 189 - [Removed](#removed-11) 190 - [v1.23.5](#v1235) 191 - [Downloads for v1.23.5](#downloads-for-v1235) 192 - [Source Code](#source-code-12) 193 - [Client Binaries](#client-binaries-12) 194 - [Server Binaries](#server-binaries-12) 195 - [Node Binaries](#node-binaries-12) 196 - [Container Images](#container-images-12) 197 - [Changelog since v1.23.4](#changelog-since-v1234) 198 - [Changes by Kind](#changes-by-kind-12) 199 - [API Change](#api-change-4) 200 - [Feature](#feature-7) 201 - [Bug or Regression](#bug-or-regression-12) 202 - [Dependencies](#dependencies-12) 203 - [Added](#added-12) 204 - [Changed](#changed-12) 205 - [Removed](#removed-12) 206 - [v1.23.4](#v1234) 207 - [Downloads for v1.23.4](#downloads-for-v1234) 208 - [Source Code](#source-code-13) 209 - [Client Binaries](#client-binaries-13) 210 - [Server Binaries](#server-binaries-13) 211 - [Node Binaries](#node-binaries-13) 212 - [Container Images](#container-images-13) 213 - [Changelog since v1.23.3](#changelog-since-v1233) 214 - [Changes by Kind](#changes-by-kind-13) 215 - [API Change](#api-change-5) 216 - [Feature](#feature-8) 217 - [Bug or Regression](#bug-or-regression-13) 218 - [Dependencies](#dependencies-13) 219 - [Added](#added-13) 220 - [Changed](#changed-13) 221 - [Removed](#removed-13) 222 - [v1.23.3](#v1233) 223 - [Downloads for v1.23.3](#downloads-for-v1233) 224 - [Source Code](#source-code-14) 225 - [Client Binaries](#client-binaries-14) 226 - [Server Binaries](#server-binaries-14) 227 - [Node Binaries](#node-binaries-14) 228 - [Container Images](#container-images-14) 229 - [Changelog since v1.23.2](#changelog-since-v1232) 230 - [Changes by Kind](#changes-by-kind-14) 231 - [Feature](#feature-9) 232 - [Bug or Regression](#bug-or-regression-14) 233 - [Dependencies](#dependencies-14) 234 - [Added](#added-14) 235 - [Changed](#changed-14) 236 - [Removed](#removed-14) 237 - [v1.23.2](#v1232) 238 - [Downloads for v1.23.2](#downloads-for-v1232) 239 - [Source Code](#source-code-15) 240 - [Client Binaries](#client-binaries-15) 241 - [Server Binaries](#server-binaries-15) 242 - [Node Binaries](#node-binaries-15) 243 - [Changelog since v1.23.1](#changelog-since-v1231) 244 - [Changes by Kind](#changes-by-kind-15) 245 - [Feature](#feature-10) 246 - [Bug or Regression](#bug-or-regression-15) 247 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-1) 248 - [Dependencies](#dependencies-15) 249 - [Added](#added-15) 250 - [Changed](#changed-15) 251 - [Removed](#removed-15) 252 - [v1.23.1](#v1231) 253 - [Downloads for v1.23.1](#downloads-for-v1231) 254 - [Source Code](#source-code-16) 255 - [Client Binaries](#client-binaries-16) 256 - [Server Binaries](#server-binaries-16) 257 - [Node Binaries](#node-binaries-16) 258 - [Changelog since v1.23.0](#changelog-since-v1230) 259 - [Changes by Kind](#changes-by-kind-16) 260 - [Feature](#feature-11) 261 - [Bug or Regression](#bug-or-regression-16) 262 - [Dependencies](#dependencies-16) 263 - [Added](#added-16) 264 - [Changed](#changed-16) 265 - [Removed](#removed-16) 266 - [v1.23.0](#v1230) 267 - [Downloads for v1.23.0](#downloads-for-v1230) 268 - [Source Code](#source-code-17) 269 - [Client Binaries](#client-binaries-17) 270 - [Server Binaries](#server-binaries-17) 271 - [Node Binaries](#node-binaries-17) 272 - [Changelog since v1.22.0](#changelog-since-v1220) 273 - [What's New (Major Themes)](#whats-new-major-themes) 274 - [Deprecation of FlexVolume](#deprecation-of-flexvolume) 275 - [Deprecation of klog specific flags](#deprecation-of-klog-specific-flags) 276 - [Software Supply Chain SLSA Level 1 Compliance in the Kubernetes Release Process](#software-supply-chain-slsa-level-1-compliance-in-the-kubernetes-release-process) 277 - [IPv4/IPv6 Dual-stack Networking graduates to GA](#ipv4ipv6-dual-stack-networking-graduates-to-ga) 278 - [HorizontalPodAutoscaler v2 graduates to GA](#horizontalpodautoscaler-v2-graduates-to-ga) 279 - [Generic Ephemeral Volume feature graduates to GA](#generic-ephemeral-volume-feature-graduates-to-ga) 280 - [Skip Volume Ownership change graduates to GA](#skip-volume-ownership-change-graduates-to-ga) 281 - [Allow CSI drivers to opt-in to volume ownership and permission change graduates to GA](#allow-csi-drivers-to-opt-in-to-volume-ownership-and-permission-change-graduates-to-ga) 282 - [PodSecurity graduates to Beta](#podsecurity-graduates-to-beta) 283 - [Container Runtime Interface (CRI) v1 is default](#container-runtime-interface-cri-v1-is-default) 284 - [Structured logging graduate to Beta](#structured-logging-graduate-to-beta) 285 - [Simplified Multi-point plugin configuration for scheduler](#simplified-multi-point-plugin-configuration-for-scheduler) 286 - [CSI Migration updates](#csi-migration-updates) 287 - [Urgent Upgrade Notes](#urgent-upgrade-notes) 288 - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade) 289 - [Known Issues](#known-issues) 290 - [Etcd v3.5.[0-2] data corruption](#etcd-v350-2-data-corruption) 291 - [Changes by Kind](#changes-by-kind-17) 292 - [Deprecation](#deprecation) 293 - [API Change](#api-change-6) 294 - [Feature](#feature-12) 295 - [Documentation](#documentation) 296 - [Failing Test](#failing-test-1) 297 - [Bug or Regression](#bug-or-regression-17) 298 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-2) 299 - [Dependencies](#dependencies-17) 300 - [Added](#added-17) 301 - [Changed](#changed-17) 302 - [Removed](#removed-17) 303 - [v1.23.0-rc.1](#v1230-rc1) 304 - [Downloads for v1.23.0-rc.1](#downloads-for-v1230-rc1) 305 - [Source Code](#source-code-18) 306 - [Client Binaries](#client-binaries-18) 307 - [Server Binaries](#server-binaries-18) 308 - [Node Binaries](#node-binaries-18) 309 - [Changelog since v1.23.0-rc.0](#changelog-since-v1230-rc0) 310 - [Changes by Kind](#changes-by-kind-18) 311 - [Bug or Regression](#bug-or-regression-18) 312 - [Dependencies](#dependencies-18) 313 - [Added](#added-18) 314 - [Changed](#changed-18) 315 - [Removed](#removed-18) 316 - [v1.23.0-rc.0](#v1230-rc0) 317 - [Downloads for v1.23.0-rc.0](#downloads-for-v1230-rc0) 318 - [Source Code](#source-code-19) 319 - [Client Binaries](#client-binaries-19) 320 - [Server Binaries](#server-binaries-19) 321 - [Node Binaries](#node-binaries-19) 322 - [Changelog since v1.23.0-beta.0](#changelog-since-v1230-beta0) 323 - [Changes by Kind](#changes-by-kind-19) 324 - [API Change](#api-change-7) 325 - [Feature](#feature-13) 326 - [Bug or Regression](#bug-or-regression-19) 327 - [Dependencies](#dependencies-19) 328 - [Added](#added-19) 329 - [Changed](#changed-19) 330 - [Removed](#removed-19) 331 - [v1.23.0-beta.0](#v1230-beta0) 332 - [Downloads for v1.23.0-beta.0](#downloads-for-v1230-beta0) 333 - [Source Code](#source-code-20) 334 - [Client Binaries](#client-binaries-20) 335 - [Server Binaries](#server-binaries-20) 336 - [Node Binaries](#node-binaries-20) 337 - [Changelog since v1.23.0-alpha.4](#changelog-since-v1230-alpha4) 338 - [Urgent Upgrade Notes](#urgent-upgrade-notes-1) 339 - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade-1) 340 - [Changes by Kind](#changes-by-kind-20) 341 - [Deprecation](#deprecation-1) 342 - [API Change](#api-change-8) 343 - [Feature](#feature-14) 344 - [Documentation](#documentation-1) 345 - [Bug or Regression](#bug-or-regression-20) 346 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-3) 347 - [Dependencies](#dependencies-20) 348 - [Added](#added-20) 349 - [Changed](#changed-20) 350 - [Removed](#removed-20) 351 - [v1.23.0-alpha.4](#v1230-alpha4) 352 - [Downloads for v1.23.0-alpha.4](#downloads-for-v1230-alpha4) 353 - [Source Code](#source-code-21) 354 - [Client Binaries](#client-binaries-21) 355 - [Server Binaries](#server-binaries-21) 356 - [Node Binaries](#node-binaries-21) 357 - [Changelog since v1.23.0-alpha.3](#changelog-since-v1230-alpha3) 358 - [Changes by Kind](#changes-by-kind-21) 359 - [Deprecation](#deprecation-2) 360 - [API Change](#api-change-9) 361 - [Feature](#feature-15) 362 - [Failing Test](#failing-test-2) 363 - [Bug or Regression](#bug-or-regression-21) 364 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-4) 365 - [Dependencies](#dependencies-21) 366 - [Added](#added-21) 367 - [Changed](#changed-21) 368 - [Removed](#removed-21) 369 - [v1.23.0-alpha.3](#v1230-alpha3) 370 - [Downloads for v1.23.0-alpha.3](#downloads-for-v1230-alpha3) 371 - [Source Code](#source-code-22) 372 - [Client Binaries](#client-binaries-22) 373 - [Server Binaries](#server-binaries-22) 374 - [Node Binaries](#node-binaries-22) 375 - [Changelog since v1.23.0-alpha.2](#changelog-since-v1230-alpha2) 376 - [Changes by Kind](#changes-by-kind-22) 377 - [Deprecation](#deprecation-3) 378 - [API Change](#api-change-10) 379 - [Feature](#feature-16) 380 - [Bug or Regression](#bug-or-regression-22) 381 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-5) 382 - [Dependencies](#dependencies-22) 383 - [Added](#added-22) 384 - [Changed](#changed-22) 385 - [Removed](#removed-22) 386 - [v1.23.0-alpha.2](#v1230-alpha2) 387 - [Downloads for v1.23.0-alpha.2](#downloads-for-v1230-alpha2) 388 - [Source Code](#source-code-23) 389 - [Client Binaries](#client-binaries-23) 390 - [Server Binaries](#server-binaries-23) 391 - [Node Binaries](#node-binaries-23) 392 - [Changelog since v1.23.0-alpha.1](#changelog-since-v1230-alpha1) 393 - [Changes by Kind](#changes-by-kind-23) 394 - [Deprecation](#deprecation-4) 395 - [API Change](#api-change-11) 396 - [Feature](#feature-17) 397 - [Documentation](#documentation-2) 398 - [Bug or Regression](#bug-or-regression-23) 399 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-6) 400 - [Dependencies](#dependencies-23) 401 - [Added](#added-23) 402 - [Changed](#changed-23) 403 - [Removed](#removed-23) 404 - [v1.23.0-alpha.1](#v1230-alpha1) 405 - [Downloads for v1.23.0-alpha.1](#downloads-for-v1230-alpha1) 406 - [Source Code](#source-code-24) 407 - [Client Binaries](#client-binaries-24) 408 - [Server Binaries](#server-binaries-24) 409 - [Node Binaries](#node-binaries-24) 410 - [Changelog since v1.22.0](#changelog-since-v1220-1) 411 - [Urgent Upgrade Notes](#urgent-upgrade-notes-2) 412 - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade-2) 413 - [Changes by Kind](#changes-by-kind-24) 414 - [Deprecation](#deprecation-5) 415 - [API Change](#api-change-12) 416 - [Feature](#feature-18) 417 - [Documentation](#documentation-3) 418 - [Bug or Regression](#bug-or-regression-24) 419 - [Other (Cleanup or Flake)](#other-cleanup-or-flake-7) 420 - [Dependencies](#dependencies-24) 421 - [Added](#added-24) 422 - [Changed](#changed-24) 423 - [Removed](#removed-24) 424 425 <!-- END MUNGE: GENERATED_TOC --> 426 427 # v1.23.17 428 429 430 ## Downloads for v1.23.17 431 432 433 434 ### Source Code 435 436 filename | sha512 hash 437 -------- | ----------- 438 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes.tar.gz) | 886292f279417ed9e63978c685df3c20ef677cd45344cce2c7dd3942a5ca1682a1c6743c4713df794b82152b059ceee437aded2eae0ab81a29320ad6eb0942d0 439 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-src.tar.gz) | f39f80416da9835ed912f04e54579ebd98b0b6e3c2cc1097abc1c98ec4d9b020f0f60ac733f323b38e6230e39ebf9d40a272885af2fbdf842a6932b051053ec5 440 441 ### Client Binaries 442 443 filename | sha512 hash 444 -------- | ----------- 445 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-darwin-amd64.tar.gz) | 5ce19fcaa82dc4794e804650835495aa49ee6edd9509536384804e9aebf29e34a152c077d25605abcac391dfcab6fa98c3b911bb8e7863193f8e62875836315c 446 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-darwin-arm64.tar.gz) | f9fea145d182921eee9bc19c5fe9051d581dbec550a7ea645eeee942ddcb41f1427951754bedf5dcfc6ea4a2f5962542306312ca82f22548bb53991bc6cfc43f 447 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-linux-386.tar.gz) | 5947ef238998131c7441c25426fc42b5a7f19dfeac0a5d85f8eddc20b1a22a85b5b34b7100e2a9139f83f84108d23bdda2220e0976efe2e7e4ec6104efd4dad4 448 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-linux-amd64.tar.gz) | 3b1ebf273e48809984cccd8f3a4c999e1f9f1b9a8f91637f4cdef29b0b74408cec3f68898c5c12ef5675f8e4d335100aba765730f0f4cf0d13ee75d28c571bf3 449 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-linux-arm.tar.gz) | 19b14ea8c14d894e5ed0fe7cb891f261ff6d81ac5bf6de96f4a516cb122ff27a6760372459c3c1f2513271c42093bd90634c1e47b8dfc2d13e66372c4638887a 450 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-linux-arm64.tar.gz) | 9585290aea3606edc2a9ba7ab3cd9dfcca20e48ea7e431ee59922015b5c05cc4c4b7aab321824a0944b111a98cbcaa9eca9998f224ddfce3158832f385490fa8 451 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-linux-ppc64le.tar.gz) | 02a344446415d789e5a5091c4e47d6f6c0f93bbb490320cce984792b2870e9dd1ef2e006d380d28f4a2e8dfa8b1693b879d25ce1c7724bee43543131cb3d1722 452 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-linux-s390x.tar.gz) | b976198ac6299eb683978921f82da27487b77c08ed6af6ab4d2c4c1e8487479911ed71c739fa64644e135b8dc2d15db0b0486a5cefe01f4daf5e4c5915088018 453 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-windows-386.tar.gz) | 20ad752499a50c6ff008d9bb379bc0584f8b776a0cb09f20da44f692df9442b41471109bc0d1aab0ceb1b58e3713744fa100cb3e88a4a9f9c76e929d89496c80 454 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-windows-amd64.tar.gz) | bf75fe667a8ddf19a8acbe9fd9e8ade9ff6ef8894eabe188fae43699f467fd3a7138f4024ab14f1f26cf7d03055cc214d93e8164a81a58183c1da310535fac00 455 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-client-windows-arm64.tar.gz) | e30ed886c242c927723f11a48864d894b3890b22a8cbab1e1a952e259ad05fb1dd7b10cefe370f81316d18ad8e0cf418af6c66cc8ee6b7cd2a44da5afc94f48d 456 457 ### Server Binaries 458 459 filename | sha512 hash 460 -------- | ----------- 461 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-server-linux-amd64.tar.gz) | a9a4b132f5b5cce0b68bd55a63d3c71564cc28e9a122787eea743cb043063d74982d798be8407ee961df8063db57d7e61ca6137b7a93b18ff650e8f1d044d712 462 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-server-linux-arm.tar.gz) | 65e75a70bae70899abb4c1bf3edaccc74b873d8808f8f85a4b29754260934b3309116a2e1b7c3e571eb278fc5fb4de13a5cce5222bcd72f092471a26fa2d472f 463 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-server-linux-arm64.tar.gz) | ffb5b4de396ad1e51fc7bf30891f2d87d9735849c39aaf492a9e0bf3c6a91c953b897be80a0c312001937d32b8bb648b3001655a752ec88939413dacf4d32074 464 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-server-linux-ppc64le.tar.gz) | eaecef5c67d88e0f9da832ad96153068addb4d9280f22d2a3c2080b0e27561eb4ec3a0bbb5febc2f86f0cb7132caf122e29fd7511d963fa6915443d4efe31573 465 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-server-linux-s390x.tar.gz) | af876a0a893e340634bd4589da00aca445781046ed7b471f834341f8266e29daa9cddcc1cacb29f9a028dc972c3d1766fdfee2ac121b7359e349a4b188343242 466 467 ### Node Binaries 468 469 filename | sha512 hash 470 -------- | ----------- 471 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-node-linux-amd64.tar.gz) | 3c123cc4e171e8755faa4a03a23234cb23024c9e29ce11914d40613191e9911c18264c944accb740bbbc976988dfae652a0829cb02460955fd33244b363a2fe1 472 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-node-linux-arm.tar.gz) | cfd08aafb09b0864b317726cb89230cdcb56806aa272ba8365c2dcf8d85b670d6beff0a11fdae7bca27fe2570f65845c6d14eda236f2d4d089e8152b062961bd 473 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-node-linux-arm64.tar.gz) | fb7664b11ea4b520b1d52664e17af8673c73b84431ffe4d6ec84e28b1a5cfcbc2348adb046eac9951b2abf4a2ead94f92e7bb81ea13d4d9fc646bc563154d22e 474 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-node-linux-ppc64le.tar.gz) | ba0774345de9427683f15efe6d7de535d47232483566346b6e4e9858fd2988eb47181e7b3383198668bc0836adeb6be73bc831abfe82874a46a0731c5eb7a69f 475 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-node-linux-s390x.tar.gz) | a8dafd03e094b8acd05ea91f5cb18bd735a97c04b7286fb69ecdc3e38f0c443a431920b89795f51850d3e8df13deae45754648e9e8df16ca3644223a069e7244 476 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.17/kubernetes-node-windows-amd64.tar.gz) | fecc305f6047044dc3d50213922e0d317b45f210bb066e442cb6ce04a9096fe6d8377d98f2b9b4adabebfa29d58a6c7a835ab9ea6d496f98973416098225243d 477 478 ### Container Images 479 480 All container images are available as manifest lists and support the described 481 architectures. It is also possible to pull a specific architecture directly by 482 adding the "-$ARCH" suffix to the container image name. 483 484 name | architectures 485 ---- | ------------- 486 [registry.k8s.io/conformance:v1.23.17](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 487 [registry.k8s.io/kube-apiserver:v1.23.17](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 488 [registry.k8s.io/kube-controller-manager:v1.23.17](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 489 [registry.k8s.io/kube-proxy:v1.23.17](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 490 [registry.k8s.io/kube-scheduler:v1.23.17](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 491 492 ## Changelog since v1.23.16 493 494 ## Changes by Kind 495 496 ### Feature 497 498 - Kubelet TCP and HTTP probes are more effective using networking resources: conntrack entries, sockets, ... 499 This is achieved by reducing the TIME-WAIT state of the connection to 1 second, instead of the defaults 60 seconds. This allows kubelet to free the socket, and free conntrack entry and ephemeral port associated. ([#115143](https://github.com/kubernetes/kubernetes/pull/115143), [@aojea](https://github.com/aojea)) [SIG Network and Node] 500 - Kubernetes is now built with Go 1.19.6 ([#115830](https://github.com/kubernetes/kubernetes/pull/115830), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 501 502 ### Bug or Regression 503 504 - Golang.org/x/net updates to v0.7.0 to fix CVE-2022-41723 ([#115790](https://github.com/kubernetes/kubernetes/pull/115790), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Security and Storage] 505 506 ## Dependencies 507 508 ### Added 509 _Nothing has changed._ 510 511 ### Changed 512 - golang.org/x/net: 1e63c2f → v0.7.0 513 - golang.org/x/sys: v0.3.0 → v0.5.0 514 - golang.org/x/term: v0.3.0 → v0.5.0 515 - golang.org/x/text: v0.5.0 → v0.7.0 516 517 ### Removed 518 _Nothing has changed._ 519 520 521 522 # v1.23.16 523 524 525 ## Downloads for v1.23.16 526 527 528 529 ### Source Code 530 531 filename | sha512 hash 532 -------- | ----------- 533 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes.tar.gz) | f4e872ef6b315607428920907a9a55a9d7546d84ca6102b79ba107d642c9878f188cb3d4331428c4f15d59cbec65ab24f8038b65202e86acdd31fb65cbe7d2ab 534 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-src.tar.gz) | edebfdbd83ba33ca46b242da35b7cd41ca57e4d218e7dbed3ad74df58cab8f2e9837f50031f9c8ab82806e55a1bed0edaffedf337c60f238d807987cc4e172b8 535 536 ### Client Binaries 537 538 filename | sha512 hash 539 -------- | ----------- 540 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-darwin-amd64.tar.gz) | 40cca087af79f67a260b93c692da6c6bbf5ab791d38ee1df33903ed8f817c0fd3bc5eb62adfe6b2f36cd209bc652b0746858201ebd7893ab50fad5bb375a3276 541 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-darwin-arm64.tar.gz) | 8bc8cc244ee5d0d578280cbb7c9e203294f5cf3a85bb42c2f98803841f2346009bb010347451cbfe1ca847fd4a89a60d7e2cc2ae43fb0401d50fa7503425be05 542 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-linux-386.tar.gz) | d5b40c11113c4758e4f7d9b645dbde1668f2c1f048a88a283640b9416219438755f7943b1156be9a9c9b1f8a3025f95065495711e36e43d6d3809e4c13405904 543 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-linux-amd64.tar.gz) | b91e9b9654cdc962390648d81d2099ee22ab8444efff18775fee7d88ddcd6b5b442ce978960211e1b2a179115c426ce1ce85fdebd4faf901695d301376f1361c 544 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-linux-arm.tar.gz) | 8d33aba4ab24ccceb5b17fb29c6a4fe53b6986124fa95d9fed7a9e770ecf9987ed87e6d316e9e898f2cb588ffd9d534bcd654014a5188c49713773997d9f62f1 545 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-linux-arm64.tar.gz) | 33c5d05b18d71095cc9e7f7c3cf0b81b3cafc1819327483d9d757da73539ee3aa20b53f9f33b4bdba299eab06ecc1fe6bcc00f231e005b1f80c5240b8c7d922f 546 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-linux-ppc64le.tar.gz) | 86c3b36db1b7ee491efa6201c64c307f0d01bc22668f9ff39bbc96ee2dde90d80142a4d7b7cc2b44891e2ab2574309d781a5c42c09a268de6815a12340a44a74 547 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-linux-s390x.tar.gz) | 294ce4d00553d720d7a862b09a153209e5de39578d52e44b2af17c104fdbe78e78671d6ee078efc22a19df935342fd22b92109ed2ff83cd930cbb966547b3860 548 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-windows-386.tar.gz) | 6c1c758fa377271a4a30c300dfa6cf24984b66e09a085f4b2e136bc368c1e1f10841cba3d5f3a4794c408a5e227227af632b8c162dccf895b835b210756a47ca 549 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-windows-amd64.tar.gz) | a407333fd7abff500f423ba8ee6b7bff4f9434943d630061d4aba9b5781c5a64ba6ecd72e0d527099247e53d6348a9fda220880d6d8890429c17ebf98f6d300e 550 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-client-windows-arm64.tar.gz) | a096749df3864ee2ea85179850b05b6afd53842390ac91161b3f14f3fceb0003a8efa5518acd52a751d206f97d2d0006a50dedb8a06caed3544d29a2811240f2 551 552 ### Server Binaries 553 554 filename | sha512 hash 555 -------- | ----------- 556 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-server-linux-amd64.tar.gz) | b9ac9a70ac41969b9532f6b22225924214c7a7a015e459642818207974e95440bc3533d353f43bce80556e25729d2227b2ff0485ca0ee7b6fc040eac948db243 557 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-server-linux-arm.tar.gz) | ce898b7a658e75034738a4fbde47306df85f2fc10ba6c6d238316b786ed0553d30d37034956167ea6456da497fe203661dd4e297f8e4013b0eb0a5ef439e7fbf 558 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-server-linux-arm64.tar.gz) | 789abee922dd618ddb9a0189341f30196614e942c41572f593b4ba0c8afa962f63defae303c5ebf067c5e26c38f61d714b0778c901ea4d07dc36141be4d91e62 559 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-server-linux-ppc64le.tar.gz) | d23d0946be48770e4bf644463fa4910f800f365231af25dd0929c9c1a8fee8d2613d7012451f697f36a905d4d63907408e03c24cd72df2648dbe2d37f8bdc4f2 560 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-server-linux-s390x.tar.gz) | a6b29082c52a9ebd562132e6ad47075a4b49bdf7c98899a47997b62463764350247d506ad9de9fc8ef031b9f5d3af17bf93472cfc507ec4010b8c697af61035c 561 562 ### Node Binaries 563 564 filename | sha512 hash 565 -------- | ----------- 566 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-node-linux-amd64.tar.gz) | 3fed26409429c87bbd60da45ba72c67576f036da16242ef92567554180f18d1960a1a2dfb525ecba915ef926f00915bf29bcd3033369f96741ba917b95d34086 567 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-node-linux-arm.tar.gz) | 514201f6925573ec0fbfe0a2519025b4ff0fc4bdda01e36cec2345a92ef432b3b3fcd7fdcdf03e7809c3faa655f143d1694bbb8c0b86f7646024f24d9a8ba929 568 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-node-linux-arm64.tar.gz) | 0e8ae7a52fbef727840cd8fa9d7ee324bf6fe7a129135aa0b2bc2dd36bae536070a61063490a2cb711319a4b2b9c06c2aefd6f0681a8752be64d30b1385a1e91 569 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-node-linux-ppc64le.tar.gz) | 80941bcdd3ab9afd65a8200dfc0c8b0133b94ee40f88196385c6bbaebece3c32ea61b961ea99ed6389483077ca74072b86aefbe2d16e1ce9fa090b1f11e3718b 570 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-node-linux-s390x.tar.gz) | 5a514a3458aa1ae8676ae07c09cea882973a82f8bc15ad43d6485618ad6509afe78d109e73e82197bb20862bb6bde6914c9e48c9d8d9b2a87d246d3eba21ca69 571 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.16/kubernetes-node-windows-amd64.tar.gz) | 711c264f937205cbe5b90903e6a18a4c7bbd0c5f7c179f635691822c78f494045f245ea215f1edb6369fe5509bb53d5be5a5f7c4644af4b85bd5ab348cdb2839 572 573 ### Container Images 574 575 All container images are available as manifest lists and support the described 576 architectures. It is also possible to pull a specific architecture directly by 577 adding the "-$ARCH" suffix to the container image name. 578 579 name | architectures 580 ---- | ------------- 581 [registry.k8s.io/conformance:v1.23.16](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 582 [registry.k8s.io/kube-apiserver:v1.23.16](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 583 [registry.k8s.io/kube-controller-manager:v1.23.16](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 584 [registry.k8s.io/kube-proxy:v1.23.16](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 585 [registry.k8s.io/kube-scheduler:v1.23.16](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 586 587 ## Changelog since v1.23.15 588 589 ## Changes by Kind 590 591 ### API Change 592 593 - Kubernetes 1.23 is now built with go1.19.4. To match behavior of previous Kubernetes 1.23 patch releases: 594 - `kube-apiserver` defaults the GOGC setting to 63, to approximate go1.17 garbage collection memory performance in heavily loaded API servers 595 - `kube-apiserver` defaults the GODEBUG x509sha1 setting to `x509sha1=1` to match go1.17 support for sha1 certificates ([#113983](https://github.com/kubernetes/kubernetes/pull/113983), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing] 596 597 ### Feature 598 599 - Kubernetes is now built with Go 1.19.5 ([#115011](https://github.com/kubernetes/kubernetes/pull/115011), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Release and Testing] 600 601 ### Bug or Regression 602 603 - Client-go: fixes potential data races retrying requests using a custom io.Reader body; with this fix, only requests with no body or with string / []byte / runtime.Object bodies can be retried ([#113933](https://github.com/kubernetes/kubernetes/pull/113933), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 604 - Do not include preemptor pod metadata in the event message ([#115025](https://github.com/kubernetes/kubernetes/pull/115025), [@mimowo](https://github.com/mimowo)) [SIG Scheduling] 605 - Failed pods associated with a job with `parallelism = 1` are recreated by the job controller honoring exponential backoff delay again. However, for jobs with `parallelism > 1`, pods might be created without exponential backoff delay. ([#115020](https://github.com/kubernetes/kubernetes/pull/115020), [@nikhita](https://github.com/nikhita)) [SIG Apps] 606 - Fixed StatefulSet to show the valid status even if the new replica creation fails. ([#112084](https://github.com/kubernetes/kubernetes/pull/112084), [@gjkim42](https://github.com/gjkim42)) [SIG Apps and Testing] 607 - Kube-apiserver: bugfix DeleteCollection API fails if request body is non-empty ([#113967](https://github.com/kubernetes/kubernetes/pull/113967), [@sxllwx](https://github.com/sxllwx)) [SIG API Machinery] 608 - Kubelet: make the image pull time more accurate in event ([#114430](https://github.com/kubernetes/kubernetes/pull/114430), [@pacoxu](https://github.com/pacoxu)) [SIG Node] 609 - [aws] Fixed a bug which reduces the number of unnecessary calls to STS in the event of assume role failures in the legacy cloud provider ([#110706](https://github.com/kubernetes/kubernetes/pull/110706), [@prateekgogia](https://github.com/prateekgogia)) [SIG Cloud Provider] 610 611 ## Dependencies 612 613 ### Added 614 _Nothing has changed._ 615 616 ### Changed 617 - github.com/yuin/goldmark: [v1.4.0 → v1.4.13](https://github.com/yuin/goldmark/compare/v1.4.0...v1.4.13) 618 - golang.org/x/crypto: 32db794 → 8634188 619 - golang.org/x/mod: v0.4.2 → 86c51ed 620 - golang.org/x/net: 491a49a → 1e63c2f 621 - golang.org/x/oauth2: 2bc19b1 → d3ed0bb 622 - golang.org/x/sync: 036812b → 886fb93 623 - golang.org/x/sys: f4d4317 → v0.3.0 624 - golang.org/x/term: 6886f2d → v0.3.0 625 - golang.org/x/text: v0.3.7 → v0.5.0 626 - golang.org/x/time: 1f47c86 → 90d013b 627 - golang.org/x/tools: d4cc65f → v0.1.12 628 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.33 → v0.0.35 629 630 ### Removed 631 _Nothing has changed._ 632 633 634 635 # v1.23.15 636 637 638 ## Downloads for v1.23.15 639 640 641 642 ### Source Code 643 644 filename | sha512 hash 645 -------- | ----------- 646 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes.tar.gz) | 75abc129db8c36f0e6c3f617bf07c404da1718f3e49ee02b61da1aca263063ed4513e344ea92507b85a418094fd1f8ac4b9d1e650bbea32024d0f134a8bbec9c 647 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-src.tar.gz) | c2987ea018c02c9d74efcc3b1ea11737427a76dd4689ec8e1975951c50bb046a507d997dd02c85fa75ac8401d4212a90d848549955288f002117f169569593aa 648 649 ### Client Binaries 650 651 filename | sha512 hash 652 -------- | ----------- 653 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-darwin-amd64.tar.gz) | 95ce601c970d1bbc14c59b5a90771c74461a32cbb62c688ee2fef29c17f6347f43439b2e16802857844dce4a6f380d07ecde5f0cfa99924cb4fceb9b67032bfd 654 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-darwin-arm64.tar.gz) | 10ae6d54f21b5520b8e870b819ee382bb87c8bff4a1d8309a33d5e18f5407d6f1d18f2b0fea360007da786d12731d91c78bbefb33abdc697d16557304a795327 655 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-linux-386.tar.gz) | 31d3ef04469842609c33f9abd55edfa5b6edb60f7f5eff2131b138f7504bbb56f45fd3c6ca7ff2362771f980a1089364b0bebb8d952f638762cf68ad58e71e05 656 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-linux-amd64.tar.gz) | 7396f8f3597bf4ea97dc2788b415934385b829fa2c91f3f7458fedd543cf9123137be118a2de46fc0a5cd80378f77ebe97bb3c3dc44bd4fbd0cd921983694287 657 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-linux-arm.tar.gz) | c3189ca8c3ca36bc345a1339be4a142c801e62f0570c5847b4bcd2da612be31392c9fbebf3ca6d6340e8d8e5e83b9b600ee87f17b3611c84af159a6107da93c7 658 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-linux-arm64.tar.gz) | 4650b7c2172b55e37f13d2eb596b37b50b33608e3017ca06cdc1e8ffb15ddf9cfc5c44a4740d285007b320d0bd37049da2b58cbb07495276d1cd01320615a617 659 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-linux-ppc64le.tar.gz) | 26d46dd1db4b4bc0cdc00055c2e434015879de97ef8a2be9a01579e202ac1f2b966392544ca1c42d22e8ae2758f98cea1b9707835915a94e990bc65413f56453 660 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-linux-s390x.tar.gz) | 0f385dd49aecb4493e059543b132ca4a9b5ac3b71002f3495bc37d390ba7866bb3a650fb26236a719f82d11edbbc6397737d58dead3dc29be282027d8b97a5a8 661 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-windows-386.tar.gz) | 3286a73cfa3e39f75524f08990148b970bdba6120548f47e4e341ca3689dc0bf5c1ab8826611475d4df3f52c708ba81db5b0c594096045d2b7024aa139a4f61a 662 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-windows-amd64.tar.gz) | 3120763fd2708a2a8b0cb875bee6e4e466a1b97cad6f66faa7ce81235925e9240d1cce30286cfcc79ffc083845d41fabd1c1d242b5dd02ec3693e77ce9500ee1 663 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-client-windows-arm64.tar.gz) | f6231faf75e31726c4e14c063469e50a249458dda29d159d1e93071be81cd14169bc757268291b5d4f5193e4de818afaa358b6c00d9e0f23ebf0b5a8c41c1018 664 665 ### Server Binaries 666 667 filename | sha512 hash 668 -------- | ----------- 669 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-server-linux-amd64.tar.gz) | a8513eaaad75ea8a05d014187e5d6a2c58b52105296d3fb69620719973e6ff49f32bbbbeb594441f0e9a21c5ff18a81982aad9712a64dc3694f0dd06f1f5cf41 670 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-server-linux-arm.tar.gz) | 60e3ac978e6b0ea9d6fc5e4f580b996fdff2c134af0c6afb7400c9962bc83340e7338d190b49da6bbd41d3c0ab15e59e6dcdca6856f8f56e8ee8287714c4b280 671 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-server-linux-arm64.tar.gz) | 287b7643b07cbe706e92cfb9078af48c58e1cb79188d1c08efe4e46b68b2ccbc8724f5eb0dbc0e1c56cea2ca9dfd5a9d2edb41e07ad46b3e2ff41fa1197834ac 672 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-server-linux-ppc64le.tar.gz) | f3a82d6ed388c104fd569314232de92f6b679d7f9a2e5b17a6acf49a1e4b3bc5e466a74cb1fe8e3ad0e5ea93ffe8624908364a380ff921fecb80aabcb80df66e 673 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-server-linux-s390x.tar.gz) | 10b5bc8b58f5b7ff15abdfc34aa59b09302134a32d4f30d8bcf5eb32df1569964d6e3b5c6b273f51580c31e406d50665e74104e32850cfd6f153629a8f789476 674 675 ### Node Binaries 676 677 filename | sha512 hash 678 -------- | ----------- 679 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-node-linux-amd64.tar.gz) | e4bc3f661d5e944143bf7b02524ced414015207e48a0605f889d4ef982a87046b16dbb681fdc10a0cc1daa8ec966a85d7e844a2e10a0d0050fbada17e8ec0cfc 680 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-node-linux-arm.tar.gz) | e4980a49f4c501f966a9c709c2de1e8fc58851de81ea7a8088a9784ef49d1efb93fb2496d3078af9cbeaf775e61ab0c5a2cfd77468aac88047e3bf52946bfe5a 681 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-node-linux-arm64.tar.gz) | 4989abc084f5441fc6261e9d9315dc49ec631b3274a3b65271df976c9562b2d59963531b95633a8cd3256cf8b8c4e8bdccb782508346961e7d463e1f4ecb6bee 682 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-node-linux-ppc64le.tar.gz) | 53619e15364e03a6c5367699a67f0acc92e1b5f0b971f35e3b752660d246369d5ae68d3c11bd400dcaedfc5906a04c684f7fdde8fbe461861c0054bde9dd7fb8 683 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-node-linux-s390x.tar.gz) | 486a14bd1fd78672cbbbe440092b5a0726385de4ad10764a18d70516404563c54ba2cc769c0df5fb3243ba3d3ca1709d65623589d6b641218ed3cf861c59cf21 684 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.15/kubernetes-node-windows-amd64.tar.gz) | 169e5df5a8170ea0f2d91840b44bcb8778afd69598295070437259a085dfa7d9d1470753826fbf82def2bd78daa5236cbf0e6f5479be3e31e9a20eb1a07b85d0 685 686 ### Container Images 687 688 All container images are available as manifest lists and support the described 689 architectures. It is also possible to pull a specific architecture directly by 690 adding the "-$ARCH" suffix to the container image name. 691 692 name | architectures 693 ---- | ------------- 694 [registry.k8s.io/conformance:v1.23.15](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 695 [registry.k8s.io/kube-apiserver:v1.23.15](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 696 [registry.k8s.io/kube-controller-manager:v1.23.15](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 697 [registry.k8s.io/kube-proxy:v1.23.15](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 698 [registry.k8s.io/kube-scheduler:v1.23.15](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 699 700 ## Changelog since v1.23.14 701 702 ## Changes by Kind 703 704 ### Feature 705 706 - Kubeadm: use the image registry registry.k8s.io instead of k8s.gcr.io for new clusters. During upgrade, migrate users to registry.k8s.io if they were using the default of k8s.gcr.io. ([#113393](https://github.com/kubernetes/kubernetes/pull/113393), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle] 707 708 ### Bug or Regression 709 710 - Fix endpoint reconciler not being able to delete the apiserver lease on shutdown ([#114153](https://github.com/kubernetes/kubernetes/pull/114153), [@aojea](https://github.com/aojea)) [SIG API Machinery] 711 - Fix for volume reconstruction of CSI ephemeral volumes ([#113347](https://github.com/kubernetes/kubernetes/pull/113347), [@dobsonj](https://github.com/dobsonj)) [SIG Node, Storage and Testing] 712 - Fix performance issue when creating large objects using SSA with fully unspecified schemas (preserveUnknownFields). ([#111914](https://github.com/kubernetes/kubernetes/pull/111914), [@aojea](https://github.com/aojea)) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] 713 714 ### Other (Cleanup or Flake) 715 716 - Kubelet now defaults to pulling the pause image from registry.k8s.io ([#114340](https://github.com/kubernetes/kubernetes/pull/114340), [@liggitt](https://github.com/liggitt)) [SIG Node] 717 718 ## Dependencies 719 720 ### Added 721 _Nothing has changed._ 722 723 ### Changed 724 - sigs.k8s.io/structured-merge-diff/v4: v4.2.1 → v4.2.3 725 726 ### Removed 727 _Nothing has changed._ 728 729 730 731 # v1.23.14 732 733 734 ## Downloads for v1.23.14 735 736 737 738 ### Source Code 739 740 filename | sha512 hash 741 -------- | ----------- 742 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes.tar.gz) | b4ef60e57a8590e428b3b9a7b871ad3918c56bde47c215f51534c1698bf75fcd479b7c0fb8e43f5e4f647e37fa6a125c152b16ecd0378d85201118ff206c1679 743 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-src.tar.gz) | 923b56a0e45c9b58d8f42cc499191fd526805eb9ab1ebf6c5a0d37b4724f1a27a5be8297f5913b3cfeaa4bfed138f6073569d58a05b60d1396bd1888bc2448ea 744 745 ### Client Binaries 746 747 filename | sha512 hash 748 -------- | ----------- 749 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-darwin-amd64.tar.gz) | 10af1dc225e95c1aedeeb0f9b955b6774bccf5dff4a3f242c26294d633b8e4ad593f1fd058cadf533d25b481ce7eb4442ad84f2c28373753650d834b9ffb0afb 750 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-darwin-arm64.tar.gz) | 2974b5237b7f99728355dc3cffd12915f2e291a08800f5faca8a802824e4dbfc2b48387f26ed909404cf518977eb9f440ea5e64b253fb34deb7b36fd2606a135 751 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-linux-386.tar.gz) | 026e0cd6076f8f2249a9829b82153762b8b918a61d72a292b7da45d24b8c1aed3007d084097b212df1e6d7e5d83581ebf8533eabae6eb376f16135899c28e677 752 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-linux-amd64.tar.gz) | 02c650f38d79065543d2d0a003a5b8c9c668bc81b4abd086b0596e964ed16a2fdbf2d16dd84d239a17a8bf7f2ca7bcc66c5daf7b96a22e9f74c0f4e22eb2c46d 753 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-linux-arm.tar.gz) | 21ea1e7fc0bc9a2ff31c06b95410047a74b66bcd88b27bf8aecbe90a17a001acb6770f6ef4fdb0599993c1eb5497bf3517b61f81b4ef645bbc168b6da0e38107 754 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-linux-arm64.tar.gz) | 0c9011fef724067e4fa81d085ff1389c6681ddc9745d2a264c5a1c7173a6a24a6dcbcd76b10fbf9e8272c0d7ef0a53eb1b51648755e499234a32deacead26704 755 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-linux-ppc64le.tar.gz) | 95201598d72abdc8582d125468b343966b588cd610d00adc93424424659791f83944581120495f87025f52fa6707ab111e4dc1ac14f0f6554aa2f863da83051c 756 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-linux-s390x.tar.gz) | 588d034bd4ba6d056cf320b1a85b3319a7824c15a37f38e0be1a1825263c043546619f39842fc1297b55881814a5207ecd02e50b0ff87745cb4af0a3b6883fc3 757 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-windows-386.tar.gz) | 0fe4d9c2177b849fec0e315dcf4d5a578de60f3838f340645b1cd80a57c2545f4d17b825de8ae6d8a16e42b769e66321c490f51f82bfade5a1250980f382dc51 758 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-windows-amd64.tar.gz) | fba076820775283a611be753b6842e6c764512fe49e648cc40666dc3d376d402479a4d00eb8c7e53fdd73609004ae2f525c2d315671b958209f3de9a0ddce447 759 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-client-windows-arm64.tar.gz) | fca4a5c44d7dd9716720181fe09828ad83187c269573a67cc09e6b524db5a0e1b7d14f2d1257cced36aff82902262abb1294884e713ac2e044975c495f6db4d3 760 761 ### Server Binaries 762 763 filename | sha512 hash 764 -------- | ----------- 765 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-server-linux-amd64.tar.gz) | f642668f7af84aea2817813ce40e12f416158e40284532ff8bc6dd721ee53ca714d1a88fbf14ddad4c4c3ca575cd196d5f6de044e41268ee7ab25aea749689dc 766 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-server-linux-arm.tar.gz) | dc1fb1486b159e9856cac0c68c9e6ad70112de94c4c9cd8b37321a03e0a2185782704933087212a39358658f9a182674a8a349d420dfdbd548dda75b73ccf424 767 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-server-linux-arm64.tar.gz) | 61616198b40bb64ab770501ddb06099cc65a9fb6cf762ec5cc7c1b31f5df69250d20228fd718398468672c6f33bf92f4aad84a0622d46b526c23c07a9027a1f6 768 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-server-linux-ppc64le.tar.gz) | b12f73b3118478a792f0bc807138e4db5425b740d5bb92f51ea11140e120c92d304006e9b21bb945c3e38ef7000fac1da3c4c4884f0bf3034e38333d11b7e4f6 769 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-server-linux-s390x.tar.gz) | c95a6a97a15181993bd463fb20e08be778ead6a55701aed37d5d69378d6de33ac2d79b98bac3ff861321b82bff495363d032dc999abe2b5a6d171fd5994c1b42 770 771 ### Node Binaries 772 773 filename | sha512 hash 774 -------- | ----------- 775 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-node-linux-amd64.tar.gz) | c65e71c583a140a4789cdec4444ab6335ed6e49d808475d6704c1ba3dc67593e087092f1a0e99c08f692667d72ddab38d7bc39c152e384aae858685530ac6763 776 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-node-linux-arm.tar.gz) | e098e856536470b4d5b9cb7d3ea7340a23db76932defd0d9d1a1002ff856199850586d0ec98d6b0cf5bb3afe62ab188f75d5c6e8db96f73eff1175c22b68099c 777 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-node-linux-arm64.tar.gz) | abf8e6232853ff11599c1735af6d0dfb55645e14903c757ebd9dc7ee77d801b7932ea39e817a9ba33964107ef5829f603f6b3ad7c739887c5d3ae85b67a88c66 778 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-node-linux-ppc64le.tar.gz) | 833733e3387859bc7abf4f217abc829095051ba116980673c57e3f24ebc4353a323d75b156b21cdb1c7c4b93ed7ef4c60cadfac5a327fae814b0f060e4330eea 779 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-node-linux-s390x.tar.gz) | 67f517e04dac3b672ed75802c0f4b547a11b23ef5306825f7e4120da9db8ad43d61621f6c098871de3a143bb516e5a7b3f6b12bdf09502b431a6163786b8c48b 780 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.14/kubernetes-node-windows-amd64.tar.gz) | d0ba80907465ca9e6c39510c13985c4be5d10dcc9e7fa984235959425faa8b5d19499a2265c99964b41fe2f8977d95eae26f7963925a39ad66c09a203cb63483 781 782 ### Container Images 783 784 All container images are available as manifest lists and support the described 785 architectures. It is also possible to pull a specific architecture directly by 786 adding the "-$ARCH" suffix to the container image name. 787 788 name | architectures 789 ---- | ------------- 790 [k8s.gcr.io/conformance:v1.23.14](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 791 [k8s.gcr.io/kube-apiserver:v1.23.14](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 792 [k8s.gcr.io/kube-controller-manager:v1.23.14](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 793 [k8s.gcr.io/kube-proxy:v1.23.14](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 794 [k8s.gcr.io/kube-scheduler:v1.23.14](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 795 796 ## Changelog since v1.23.13 797 798 ## Important Security Information 799 800 This release contains changes that address the following vulnerabilities: 801 802 ### CVE-2022-3162: Unauthorized read of Custom Resources 803 804 A security issue was discovered in Kubernetes where users authorized to list or watch one type of namespaced custom resource cluster-wide can read custom resources of a different type in the same API group they are not authorized to read. 805 806 **Affected Versions**: 807 - kube-apiserver v1.25.0 - v1.25.3 808 - kube-apiserver v1.24.0 - v1.24.7 809 - kube-apiserver v1.23.0 - v1.23.13 810 - kube-apiserver v1.22.0 - v1.22.15 811 - kube-apiserver <= v1.21.? 812 813 **Fixed Versions**: 814 - kube-apiserver v1.25.4 815 - kube-apiserver v1.24.8 816 - kube-apiserver v1.23.14 817 - kube-apiserver v1.22.16 818 819 This vulnerability was reported by Richard Turnbull of NCC Group as part of the Kubernetes Audit 820 821 822 **CVSS Rating:** Medium (6.5) [CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N](https://www.first.org/cvss/calculator/3.0#CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N) 823 824 ### CVE-2022-3294: Node address isn't always verified when proxying 825 826 A security issue was discovered in Kubernetes where users may have access to secure endpoints in the control plane network. Kubernetes clusters are only affected if an untrusted user can to modify Node objects and send requests proxying through them. 827 828 Kubernetes supports node proxying, which allows clients of kube-apiserver to access endpoints of a Kubelet to establish connections to Pods, retrieve container logs, and more. While Kubernetes already validates the proxying address for Nodes, a bug in kube-apiserver made it possible to bypass this validation. Bypassing this validation could allow authenticated requests destined for Nodes to be redirected to the API Server through its private network. 829 830 The merged fix enforces validation against the proxying address for a Node. In some cases, the fix can break clients that depend on the `nodes/proxy` subresource, specifically if a kubelet advertises a localhost or link-local address to the Kubernetes control plane. Configuring an egress proxy for egress to the cluster network can also mitigate this vulnerability. 831 832 **Affected Versions**: 833 - kube-apiserver v1.25.0 - v1.25.3 834 - kube-apiserver v1.24.0 - v1.24.7 835 - kube-apiserver v1.23.0 - v1.23.13 836 - kube-apiserver v1.22.0 - v1.22.15 837 - kube-apiserver <= v1.21.? 838 839 **Fixed Versions**: 840 - kube-apiserver v1.25.4 841 - kube-apiserver v1.24.8 842 - kube-apiserver v1.23.14 843 - kube-apiserver v1.22.16 844 845 This vulnerability was reported by Yuval Avrahami of Palo Alto Networks 846 847 848 **CVSS Rating:** Medium (6.6) [CVSS:3.1/AV:N/AC:H/PR:H/UI:N/S:U/C:H/I:H/A:H](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:H/PR:H/UI:N/S:U/C:H/I:H/A:H) 849 850 ## Changes by Kind 851 852 ### API Change 853 854 - Make STS available replicas optional again, ([#109241](https://github.com/kubernetes/kubernetes/pull/109241), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG API Machinery and Apps] 855 - Make STS available replicas optional again. ([#113122](https://github.com/kubernetes/kubernetes/pull/113122), [@ashrayjain](https://github.com/ashrayjain)) [SIG Apps] 856 - Protobuf serialization of metav1.MicroTime timestamps (used in `Lease` and `Event` API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd. ([#111936](https://github.com/kubernetes/kubernetes/pull/111936), [@haoruan](https://github.com/haoruan)) [SIG API Machinery] 857 858 ### Bug or Regression 859 860 - Consider only plugin directory and not entire kubelet root when cleaning up mounts ([#112921](https://github.com/kubernetes/kubernetes/pull/112921), [@mattcary](https://github.com/mattcary)) [SIG Storage] 861 - Etcd: Update to v3.5.5 ([#113100](https://github.com/kubernetes/kubernetes/pull/113100), [@mk46](https://github.com/mk46)) [SIG API Machinery, Cloud Provider, Cluster Lifecycle and Testing] 862 - Fixed a bug where a change in the `appProtocol` for a Service did not trigger a load balancer update. ([#113033](https://github.com/kubernetes/kubernetes/pull/113033), [@MartinForReal](https://github.com/MartinForReal)) [SIG Cloud Provider and Network] 863 - Kube-proxy, will restart in case it detects that the Node assigned pod.Spec.PodCIDRs have changed ([#113258](https://github.com/kubernetes/kubernetes/pull/113258), [@code-elinka](https://github.com/code-elinka)) [SIG Network] 864 - Kubelet no longer reports terminated container metrics from cAdvisor ([#112964](https://github.com/kubernetes/kubernetes/pull/112964), [@bobbypage](https://github.com/bobbypage)) [SIG Node] 865 - Kubelet: fix GetAllocatableCPUs method in cpumanager ([#113422](https://github.com/kubernetes/kubernetes/pull/113422), [@Garrybest](https://github.com/Garrybest)) [SIG Node] 866 - Pod logs using --timestamps are not broken up with timestamps anymore. ([#113517](https://github.com/kubernetes/kubernetes/pull/113517), [@rphillips](https://github.com/rphillips)) [SIG Node] 867 868 ## Dependencies 869 870 ### Added 871 _Nothing has changed._ 872 873 ### Changed 874 - github.com/stretchr/objx: [v0.2.0 → v0.4.0](https://github.com/stretchr/objx/compare/v0.2.0...v0.4.0) 875 - github.com/stretchr/testify: [v1.7.0 → v1.8.0](https://github.com/stretchr/testify/compare/v1.7.0...v1.8.0) 876 - go.uber.org/goleak: v1.1.10 → v1.2.0 877 - gopkg.in/yaml.v3: 496545a → v3.0.1 878 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.30 → v0.0.33 879 880 ### Removed 881 _Nothing has changed._ 882 883 884 885 # v1.23.13 886 887 888 ## Downloads for v1.23.13 889 890 891 892 ### Source Code 893 894 filename | sha512 hash 895 -------- | ----------- 896 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes.tar.gz) | 0995ad8ee7e2a8733db5287f9dedcfa565cc7782f83a633459718a155ae60b18dd8a6978e134abdecb05bdfd28c85744e402c53e54fd1ec7332f627b2e76c2fa 897 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-src.tar.gz) | 0e3129af29a28c96528f4842caa9234dcd198d9b1712cdf262581a535fa851013ce31c6543014145bb984c8994673bc73f88bbc83ea7142bc92737b81df8bae3 898 899 ### Client Binaries 900 901 filename | sha512 hash 902 -------- | ----------- 903 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-darwin-amd64.tar.gz) | 1f94745e78d8ee50725d5145d4eac928a7ae481e51758ed259232013f245961a9a54f106b556330d710a7d8c45e7c13a304fec6f56f8c5ce8e959e204efb6b27 904 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-darwin-arm64.tar.gz) | 255843cdf86ba0a18b60d8dd5b515fa9a3225119875e9c6a3f7a128d1a0545d92c28abf8d1e2c02194f3f047417ade6752c791bfbc1469e566d305122522f60e 905 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-linux-386.tar.gz) | 4076aa1723fff8d1d9a455159994ee002855bec419c4e56d894ee95dbf527a222236bbd65e711aa967df185bdcd71d9ac3bd6dd0143cdc18c80a28ec09e675e2 906 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-linux-amd64.tar.gz) | 5dee83bd9ef3c8ce3923c7f8fa3a6cbbaab81158ae065f0721117debdbd6412cb2f091eda9e9cbcc7637533c0f328228a9714f56de303bbe61c93b94398a91f7 907 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-linux-arm.tar.gz) | 2c5026cb1204e4a3a78280c837193bed7cd1287c5053ab83f3660762e18168c5b3896709d1a3f18cf66f92b359b258b1fcc4bc9acea651531bd7fca5fa991e9d 908 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-linux-arm64.tar.gz) | f0f9505bfdf6e9b89c995797cec522dedfec34b1880a52be042c81a797ddcf080e27f6af366944247ce5edc0757aaa9f67425840bedad276e9e18180c4363334 909 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-linux-ppc64le.tar.gz) | ebd37318e83e42d8a1833598b6f31ee28d358987e5727e0bb0e9ce4e4de7c26c299a26beb4f857d852e04db569c92750f44061239493bcf040b403b41b3e865b 910 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-linux-s390x.tar.gz) | 037d6b51e4b0e7011048087940637b3653034ad22219c281b49ee76a2e7194b79502a9f4a299349772b68b3b256570a6ce7e44896746e761c606ef04e104453b 911 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-windows-386.tar.gz) | 1e938c7b44a4093d84b5370c51590105d64b5283c3543bcb579252733b062db613c5203fd336de621442963a1dd4f51bf4799289e98fe5e09f953065e503255f 912 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-windows-amd64.tar.gz) | 0f2ed66cc2c2471c3574320654b19e8894a7ee42e7e1cb9b579119abb3f9dc82087acf41b060627e3ddf0aaae6a0ed04cbbe740cbbef26eaa754cce715ae32de 913 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-client-windows-arm64.tar.gz) | 02032b45f01c2b2a1ebbe48aace0f44297c677f64e122c2933f0b66ecb0ca48c13795097cce9007efa6f70f2dbdac42b6a08219ecebe29404a728202ebdabddc 914 915 ### Server Binaries 916 917 filename | sha512 hash 918 -------- | ----------- 919 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-server-linux-amd64.tar.gz) | 59b8bb9a28799c2f07c961bceac6b600c443df30e0a5051950f784f031c69dd7deb46555a734696a262a95d7d1d0b5e1623440ca4017806896d06758a3a08d8b 920 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-server-linux-arm.tar.gz) | 2061a03b768b9e24055dd7280e1695717c5eba32cecbdd8c96be0755f95907505106bbd0e24d8cb9098c5d9cd1aa6f70529550a411317cf749fd89457923b33f 921 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-server-linux-arm64.tar.gz) | b4ca99506f8efa6232e7a4a7d6d37973b317fb053f21b73a5bebe2f2e685cafca75a29345a67ffe41b631e5af9a7c5e2625e6113b0eebdd2781547728c976a70 922 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-server-linux-ppc64le.tar.gz) | fb3d036fde212363bbcb5c8e04cca76eeea6ec90e3513a9f2f4cbef4655548b4e061143be9c6e0e17b2dc496ae1990847a11c948b7d72d9e9fd0555d7a4df5a7 923 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-server-linux-s390x.tar.gz) | 62f09c403cb97ddb536a67ec4816846e9b7fa136faa8958831ec980d4755b2b88483292f705ca12c0c7ced0e3e3f0daa811835ff578c8d31b24c25389c09033e 924 925 ### Node Binaries 926 927 filename | sha512 hash 928 -------- | ----------- 929 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-node-linux-amd64.tar.gz) | 97167211801a737b52f766a2fd7b0afddacac22dbf4b03cbbeae734b2a7e1243f8e5d715c84f4f8dfa862c62892f01a42d2561c96386bf2283dd21e64bb91a72 930 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-node-linux-arm.tar.gz) | f1e75a62c8360296cd4bae498c46721e44f2219322f7361ca9d32a0026a5c9f797c99c3b34df2f985466aa11eaacda425ec6aab9f6368c426860efcd0496af5c 931 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-node-linux-arm64.tar.gz) | 86fb0f9cea876a7e8331ac82de1feebadca56accc3495c30c558b5851e4242a9c9fc8420c7e23c2db734b6ef8134cc0455dfb7a67798838e921c3120936386c9 932 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-node-linux-ppc64le.tar.gz) | 997190a1d39a0a2684d65922ef583e72eff9fe5c77011f3d1edd837ee7ff4ce4aa1f898b9e2f1acba4d239b2f3bf33f874ace28105b2b51825b57f699309622d 933 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-node-linux-s390x.tar.gz) | 3210ae564dc1066c70ea2bfbb6b961290fea748db8169d3ac9a00cd3082b24edbebe0b5fe0fbb276f794360ff82c502209c7c155cb2d6e58eb82a95d5fd829cb 934 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.13/kubernetes-node-windows-amd64.tar.gz) | 2cdac85dc652cc0dfde4556343e96747aefae678b14324b4eef7adfc9a9bfc2d6e7b3c9801da3ccdf9d18a52282edab054c2f0e1f6a7a06e3845c9915100195f 935 936 ### Container Images 937 938 All container images are available as manifest lists and support the described 939 architectures. It is also possible to pull a specific architecture directly by 940 adding the "-$ARCH" suffix to the container image name. 941 942 name | architectures 943 ---- | ------------- 944 [k8s.gcr.io/conformance:v1.23.13](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 945 [k8s.gcr.io/kube-apiserver:v1.23.13](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 946 [k8s.gcr.io/kube-controller-manager:v1.23.13](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 947 [k8s.gcr.io/kube-proxy:v1.23.13](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 948 [k8s.gcr.io/kube-scheduler:v1.23.13](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 949 950 ## Changelog since v1.23.12 951 952 ## Changes by Kind 953 954 ### Bug or Regression 955 956 - Azure: Skip "instance not found" error for LB backend address pools ([#111428](https://github.com/kubernetes/kubernetes/pull/111428), [@lzhecheng](https://github.com/lzhecheng)) [SIG Cloud Provider] 957 - Fix list cost estimation in Priority and Fairness for list requests with metadata.name specified. ([#112557](https://github.com/kubernetes/kubernetes/pull/112557), [@marseel](https://github.com/marseel)) [SIG API Machinery] 958 - Fixes an issue affecting log rotation with CRI runtimes where kubelet tries to compress Windows logs before closing file handles ([#112483](https://github.com/kubernetes/kubernetes/pull/112483), [@ibabou](https://github.com/ibabou)) [SIG Node and Windows] 959 - For raw block CSI volumes on Kubernetes, kubelet was incorrectly calling CSI NodeStageVolume for every single "map" (i.e. raw block "mount") operation for a volume already attached to the node. This PR ensures it is only called once per volume per node. ([#112403](https://github.com/kubernetes/kubernetes/pull/112403), [@akankshakumari393](https://github.com/akankshakumari393)) [SIG Storage] 960 - Make sure auto-mounted subpath mount source is already mounted ([#112499](https://github.com/kubernetes/kubernetes/pull/112499), [@andyzhangx](https://github.com/andyzhangx)) [SIG Storage] 961 962 ## Dependencies 963 964 ### Added 965 _Nothing has changed._ 966 967 ### Changed 968 _Nothing has changed._ 969 970 ### Removed 971 _Nothing has changed._ 972 973 974 975 # v1.23.12 976 977 978 ## Downloads for v1.23.12 979 980 981 982 ### Source Code 983 984 filename | sha512 hash 985 -------- | ----------- 986 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes.tar.gz) | c6ef86c6510a6aaff54c36190b22161d910b6ae6d9a1f0304094112ead3ee25bdbf6349cff9d5f926b287b4f75aba7befb8898d31c87f1ba19a687183e4ada09 987 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-src.tar.gz) | 5736949e8ea5dbea4c7a3ff332d4d69dca6db558c4337f189fcb0e0d712e5bddc18d580d1b61f979acd4262369a22d9e6c6c5ca36e62e1d0f5a3f51225832e9c 988 989 ### Client Binaries 990 991 filename | sha512 hash 992 -------- | ----------- 993 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-darwin-amd64.tar.gz) | eacd137e88441b56b827460eb949a01f72dbc975a62b941b3d8093d168c191f0c59a6eddc7d27165071ba827b87d1d6ae80c372f1b1bf3a3462fe55fe6946136 994 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-darwin-arm64.tar.gz) | 6c4ef41a6c324d28aef169b320552acd40d3cd64165c99b213b5834b64977cc6b2d153c50be576fc9d196303431b2a67cf53f0bb812ad7f50c06f0f6ec873abd 995 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-linux-386.tar.gz) | 1d045d962c97bc064eae749981f60ecab48c38966dd4fbb69dc1fb861d5112360b5deae6e5ddab586cf11c574f1f1ba16c00d98557dc835a8d0521ac79265354 996 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-linux-amd64.tar.gz) | 55478ab38a23ffdeca0b4e9a45f75431d8bf18f6ad08f7f2a1151be2b92479abab590e526ccbc7196f737d52b85382a01c7b8fae0af186c5a6ce4baf2e4e03f4 997 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-linux-arm.tar.gz) | c80b390cbd3039b90ac956268eac321448264b3ab8e753e6651933b7a26bfb4c0f50717f2ca3d8298b175165bb56133c5d7305c34bfa7838e4c029cb05868168 998 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-linux-arm64.tar.gz) | 1abda0aa71ecddd6a4128bf1985070490cb83d16d2c5ebed257c312a764bccd71f54ea67a21c728a4cf8675ff5dfb8525709ab4e553aaca9ddcd448c26265a31 999 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-linux-ppc64le.tar.gz) | 5747070f19780cfe1b2fc9b38f004ce542136cf637cd4ab70de57ae87c6576f5d5efe58d4d8fe6870a224f26775f0865760b03cf06773c9c11d6503bbfa5ee64 1000 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-linux-s390x.tar.gz) | fff1e69c75b64be5269afc1f801d8bf7e33343f386c9d932c4d96ec3271040793df0ce185260429c45f6cf9fbc025a2604cbe12ce01c9fe917a1684ad32398ce 1001 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-windows-386.tar.gz) | 02851aaf61e8ffc2834d9baf825b7411ff25663a94778029e629142d296da97d1ab42e4deb6e12bd23f0bc09313ae477b192491d0d369d9136ad03c8f690d4ff 1002 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-windows-amd64.tar.gz) | 95d47f3888f803a88f759c01f25d9b410a8534f745c79f925337b1d636734de71822859a427f1b5bc0d361ea391d397132031951602c984f5d4d56c61da483cf 1003 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-client-windows-arm64.tar.gz) | ae333672d0caee1b58d71514e4734ee7a2ef8851d2fa674c6af6947c7caadace3ab1d12d4852f377ac2e23a0ec95c28fb95b1e46601aa88f5e1ead2d9e19a1dc 1004 1005 ### Server Binaries 1006 1007 filename | sha512 hash 1008 -------- | ----------- 1009 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-server-linux-amd64.tar.gz) | 6e80857d208a72387f66b548a067268042b465712df6c2babc4fc5e2ac9349a3754a32cc44af5c244d23e34b0ceb9c973fe9a59fe51f558bb786c993a03a8a47 1010 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-server-linux-arm.tar.gz) | 6dc1d09f58e4c615eea96e754449932ddffd2deac540dc0c1dc657fa1650dddce5c7364ca67b68288afa1176a76ead53baecf95abd4dbcaff93e8e4b09c1a0c2 1011 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-server-linux-arm64.tar.gz) | 506303e92dca80821259a1842131b536cacafd0fe63d48bab0348c4fb6f5ca86456605bfb30a0f2993b7ab4a5d48b25eb1dabf8cc5eb4be91743343808db833c 1012 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-server-linux-ppc64le.tar.gz) | 0aad929cdd11e2494baa46bd69256086281e98f5f76beb50b21f3494b3a24da607a6f1bd87fa960c36e6ff2ffd39a23a7cf342bffa337b12ca492c1c6db5b61c 1013 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-server-linux-s390x.tar.gz) | 1a02255daddfd3c510f27bb480062493d8b0336136ec52e74b130362d7abe006193219cba115ec8296fc79f80a2c544e0fcc8a655cd7f38c3736af1f28b1c1e5 1014 1015 ### Node Binaries 1016 1017 filename | sha512 hash 1018 -------- | ----------- 1019 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-node-linux-amd64.tar.gz) | e35f9f226a214ee977deb467e60f6724cb626eed6a806c2e1c707f23f8bd6aac59d315dd3dab6d706889c4fb5f48af5919bfc606e01e6281ffa8c63b29c8bd62 1020 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-node-linux-arm.tar.gz) | b808f29dbfeb825adc0eaf799e933286c47f0978d642b240737bfd9cbb36cb40f0499a31d3b7c2201f8e1cbafd29f4d5edda02c79df1952b0079354839532ff1 1021 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-node-linux-arm64.tar.gz) | 3f7e085d8b0ed71128752beb2422438256443329a9a60812657149751b6a7e68382701472f2994fa23d28d7d00be3de7991080b9928576a72aefe0dcc29644aa 1022 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-node-linux-ppc64le.tar.gz) | 4e4c00160811404bba2c3581cc61351ab79e90d697c886ca5b0215eec54353c474915c2b4274b48326f563b14c01ef5c50249a921335b5171c4d7582eff1658c 1023 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-node-linux-s390x.tar.gz) | 701c8d8bf175cd89b670082d88db186d85c9cb68774c297e8dc3ed576475562f02e8e2bce7cb440d35d274139762d1814045b35f021e805bf0d1ad3ecbee50d9 1024 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.12/kubernetes-node-windows-amd64.tar.gz) | 3f8f84063803add6281b5afe883c4b7ff594111dea89156a4fd9ad66f8e9e6c089cf167f3f57e4903d58d7c5b18f7c4ad5684a590c66581fbabfba6ef44232aa 1025 1026 ### Container Images 1027 1028 All container images are available as manifest lists and support the described 1029 architectures. It is also possible to pull a specific architecture directly by 1030 adding the "-$ARCH" suffix to the container image name. 1031 1032 name | architectures 1033 ---- | ------------- 1034 [k8s.gcr.io/conformance:v1.23.12](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1035 [k8s.gcr.io/kube-apiserver:v1.23.12](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1036 [k8s.gcr.io/kube-controller-manager:v1.23.12](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1037 [k8s.gcr.io/kube-proxy:v1.23.12](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1038 [k8s.gcr.io/kube-scheduler:v1.23.12](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1039 1040 ## Changelog since v1.23.11 1041 1042 ## Changes by Kind 1043 1044 ### Bug or Regression 1045 1046 - Kube-apiserver: gzip compression switched from level 4 to level 1 to improve large list call latencies in exchange for higher network bandwidth usage (10-50% higher). This increases the headroom before very large unpaged list calls exceed request timeout limits. ([#112400](https://github.com/kubernetes/kubernetes/pull/112400), [@shyamjvs](https://github.com/shyamjvs)) [SIG API Machinery] 1047 - Kube-apiserver: resolved a regression that treated `304 Not Modified` responses from aggregated API servers as internal errors ([#112529](https://github.com/kubernetes/kubernetes/pull/112529), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 1048 - Kubeadm: allow RSA and ECDSA format keys in preflight check ([#112536](https://github.com/kubernetes/kubernetes/pull/112536), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] 1049 1050 ## Dependencies 1051 1052 ### Added 1053 _Nothing has changed._ 1054 1055 ### Changed 1056 _Nothing has changed._ 1057 1058 ### Removed 1059 _Nothing has changed._ 1060 1061 1062 1063 # v1.23.11 1064 1065 1066 ## Downloads for v1.23.11 1067 1068 1069 1070 ### Source Code 1071 1072 filename | sha512 hash 1073 -------- | ----------- 1074 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes.tar.gz) | 90fb263651c00d4b801feeed82446e0bfd69a42e804fdc98a08cf83c8f6fab24712d88b812bc8f70f6ffd0d33fc03099e889ec1cb141ef423c1455a59905a51a 1075 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-src.tar.gz) | e25ba52f86f8fc1afc88e4ed814a75761b2ba85f5df27c96a4fca5c26358dee8e53cde914197d2884e69155f5b45bb09e7d1a1c340f35ba44e15bc958bf5cea3 1076 1077 ### Client Binaries 1078 1079 filename | sha512 hash 1080 -------- | ----------- 1081 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-darwin-amd64.tar.gz) | 9991740791a6ff7f1ca7de7869cdc3d9c3ba24a2ed57ed38213a17d714954db3d746b6ca004edd8ec869ead0423716b52fd97c24177a2284ff293fbdc2c50913 1082 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-darwin-arm64.tar.gz) | d986a0ade1a9ef3200319d1b7b5445a7980e54e9febe7d3ad35d3b1fb3c5c1bd04e6d69727de7d1303cda9ac8b47f154270d63edf323d9a9065b0d4f0deda8bb 1083 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-linux-386.tar.gz) | 291bd7e7c63618d48a58f17efc07f815d554efc61167a908f44e494d58928d408bc1772c1c231d4ecb41a73e1a2fa34e836691ced24ff16b5614c621dddc3bca 1084 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-linux-amd64.tar.gz) | 7f4e290d5057b5d8376690d6ffa79e45217228df94fb23061b1be0adc71a9a573009058654dba6b58c5deaa457080e3a31121ea860e47f480f1a4c042e55f979 1085 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-linux-arm.tar.gz) | c625bc6d1c1f7a454116fc7397ec7a65674aff022166eb06319e45c4bfd1297d1a218e7581bea180201e6bc6232cf6110c5a211201517108cd3164151869eac8 1086 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-linux-arm64.tar.gz) | eb3c01b02b5ddc5d8f40ec401cdb81916711bed8ec49578ad1ff32f994638c9f5786032d24107b41879a4242415685174e3c94d0ee7496087b64c12f654e66e7 1087 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-linux-ppc64le.tar.gz) | e3f2993fca11589eaf27d1ee2ba513dd349e577677fc3d8b357361ecb3fa1a99e911865ff88caccaa75c2340e3535bbcaee20592aedbe53da540e79c2e2ab845 1088 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-linux-s390x.tar.gz) | 63a3dcd8f24754df3c5a2f5c970ce22d75753d027ac32da1c5a9d6bf60033e9093bf233df4d7594536e3419a6673cbf8ee361896adbea8078e895bd37254f74f 1089 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-windows-386.tar.gz) | 1ff486a460ce6d5526c7f73f8fb6e87525c139aaf935787b6decaa00095dd1e49a5ec1126007926f737394c57b19f10de3480c9c513d728ee677f688c8998066 1090 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-windows-amd64.tar.gz) | 9406aece47869024872b912d3ace880e016491ec709e28dc0e5019999782a93ec736b8af8cc453f5b5bb92cc801492056f60494be4a4d75314a73eb9dccd261c 1091 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-client-windows-arm64.tar.gz) | 9e8493a3fca2d1003092a9d8c078543dfa4e6492c98daf6b84c7ec7cb7bf8e5bdc88083f815a42903c1c2cdc8a0ef4c2220aa217c330597ca1e9c8171314846f 1092 1093 ### Server Binaries 1094 1095 filename | sha512 hash 1096 -------- | ----------- 1097 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-server-linux-amd64.tar.gz) | 767eec5ae28f8c0151379236e3f46870573964af4bf68ff38da28fc900c057d521bd6e2ca79458a95acdc6062069928f5a246ad69aff2e751f772ae9baec548a 1098 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-server-linux-arm.tar.gz) | 4cbba22196d0002494d785f03138a6b1f747c4d00c686c9a52b61fe73e57fb28ac9314edd44d6136b3c82a457b5059f267c7dfc0270e55c7ff3ed8305cee850b 1099 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-server-linux-arm64.tar.gz) | d69a5a95a05532b19cef65a020e6b8594f63b5d0fa2d29768c3c633a4170ac69f2b41e98aba1a4039b5f500d7719ad84f76010e229b296fd89ded9ac21298735 1100 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-server-linux-ppc64le.tar.gz) | 9c9ba9a27567b5ca253bee66a65e978783ce02a56d13312e933ce829dba1c7d6c00648e552ea647dccbd1f858fb0d90842a7acabecea47be8248f3e46af5d043 1101 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-server-linux-s390x.tar.gz) | b0b32c62216b18a8456788c6763eeac2c443e7c3781b29fcb356025d1c836c248c3f157ff65a94412b3b02306196259411e3f2e49a888b5f233d176b9e69541b 1102 1103 ### Node Binaries 1104 1105 filename | sha512 hash 1106 -------- | ----------- 1107 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-node-linux-amd64.tar.gz) | aace59e2477138f50799f5d536ae2370b1cb76425f0411d5022fc9ee438715be6ce27b8e6c400ede704ed2d6b599d13fb12b15eed272a4c73cec60ff08857747 1108 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-node-linux-arm.tar.gz) | 044216a04fb7c995038d1f36237fb268396621aabbee7a9c0ac492ab6e70f697b80ff833b1a4aad7f32cdbd35dd90e6931028747c73e04adec4eb786a658140c 1109 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-node-linux-arm64.tar.gz) | b31cdddeaffec6618cb1826b28f73e657bbd6b272801e4d0399ef6a5db1ea0afcd9fea050ca0a6aaa2714eeda573d0cf9441c462f0492777f07ac94d46aef8b9 1110 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-node-linux-ppc64le.tar.gz) | bdb3ddde36fc29374d5f16bf7ee078aab94f20b0b4b356eb6ceba5088d5de694f26acf79c82a7758ed0976ae754a4a7b6122e31cb0cf66a0cf3a3d1e971bc88c 1111 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-node-linux-s390x.tar.gz) | 170f42796052cb451793c42757e3e8fe69d200e20372edcfaecd42403b8a0efd7571c2456d6a21bc2c3a35c7b0c6ebcab6f83d21a772b6522a36f3d4b1c8d0ab 1112 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.11/kubernetes-node-windows-amd64.tar.gz) | 3bb5daf6403bcc88a32470feac9a248d2a0a39f21375f5b0cc5fe081062061872d4269b4d2d977eb74bf6259cc1908e1bd138bb7eeb95d373e337f85979da1b0 1113 1114 ### Container Images 1115 1116 All container images are available as manifest lists and support the described 1117 architectures. It is also possible to pull a specific architecture directly by 1118 adding the "-$ARCH" suffix to the container image name. 1119 1120 name | architectures 1121 ---- | ------------- 1122 [k8s.gcr.io/conformance:v1.23.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1123 [k8s.gcr.io/kube-apiserver:v1.23.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1124 [k8s.gcr.io/kube-controller-manager:v1.23.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1125 [k8s.gcr.io/kube-proxy:v1.23.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1126 [k8s.gcr.io/kube-scheduler:v1.23.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1127 1128 ## Changelog since v1.23.10 1129 1130 ## Important Security Information 1131 1132 This release contains changes that address the following vulnerabilities: 1133 1134 ### CVE-2022-3172: Aggregated API server can cause clients to be redirected (SSRF) 1135 1136 A security issue was discovered in kube-apiserver that could allow an attacker controlled aggregated API server to redirect client traffic to any URL. This could lead to the client performing unexpected actions as well as leaking the client's credentials to third parties. 1137 1138 There is no mitigation from this issue. Cluster admins should take care to secure aggregated API servers and should not grant access to mutate `APIService`s to untrusted parties. 1139 1140 **Affected Versions**: 1141 - kube-apiserver v1.25.0 1142 - kube-apiserver v1.24.0 - v1.24.4 1143 - kube-apiserver v1.23.0 - v1.23.10 1144 - kube-apiserver v1.22.0 - v1.22.14 1145 - kube-apiserver <= v1.21.? 1146 1147 **Fixed Versions**: 1148 - kube-apiserver v1.25.1 1149 - kube-apiserver v1.24.5 1150 - kube-apiserver v1.23.11 1151 - kube-apiserver v1.22.14 1152 1153 This vulnerability was reported by Nicolas Joly & Weinong Wang from Microsoft 1154 1155 1156 **CVSS Rating:** Medium (5.1) [CVSS:3.1/AV:N/AC:H/PR:H/UI:R/S:C/C:L/I:L/A:L](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:H/PR:H/UI:R/S:C/C:L/I:L/A:L) 1157 1158 ### CVE-2021-25749: `runAsNonRoot` logic bypass for Windows containers 1159 1160 A security issue was discovered in Kubernetes that could allow Windows workloads to run as `ContainerAdministrator` even when those workloads set the `runAsNonRoot` option to `true `. 1161 1162 This issue has been rated low and assigned CVE-2021-25749 1163 1164 **Am I vulnerable?** 1165 1166 All Kubernetes clusters with following versions, running Windows workloads with `runAsNonRoot` are impacted 1167 1168 **Affected Versions**: 1169 1170 - kubelet v1.20 - v1.21 1171 - kubelet v1.22.0 - v1.22.13 1172 - kubelet v1.23.0 - v1.23.10 1173 - kubelet v1.24.0 - v1.24.4 1174 1175 **How do I mitigate this vulnerability?** 1176 1177 There are no known mitigations to this vulnerability. 1178 1179 **Fixed Versions**: 1180 1181 - kubelet v1.22.14 1182 - kubelet v1.23.11 1183 - kubelet v1.24.5 1184 - kubelet v1.25.0 1185 1186 1187 To upgrade, refer to this documentation _For core Kubernetes:_ https://kubernetes.io/docs/tasks/administer-cluster/cluster-management/#upgrading-a-cluster 1188 1189 **Detection**: 1190 1191 Kubernetes Audit logs may indicate if the user name was misspelled to bypass the restriction placed on which user is a pod allowed to run as. 1192 1193 If you find evidence that this vulnerability has been exploited, please contact security@kubernetes.io 1194 1195 **Additional Details**: 1196 1197 See the GitHub issue for more details: https://github.com/kubernetes/kubernetes/issues/112192 1198 1199 **Acknowledgements**: 1200 1201 This vulnerability was reported and fixed by Mark Rosetti (@marosset) 1202 1203 1204 **CVSS Rating:** Low (3.4) [CVSS:3.1/AV:A/AC:H/PR:H/UI:R/S:U/C:L/I:L/A:L/E:P/RL:O/RC:C](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:A/AC:H/PR:H/UI:R/S:U/C:L/I:L/A:L/E:P/RL:O/RC:C) 1205 1206 ## Changes by Kind 1207 1208 ### Bug or Regression 1209 1210 - Fix an ephemeral port exhaustion bug caused by improper connection management that occurred when a large number of objects were handled by kubectl while exec auth was in use. ([#112338](https://github.com/kubernetes/kubernetes/pull/112338), [@enj](https://github.com/enj)) [SIG API Machinery and Auth] 1211 - Fix problem in updating VolumeAttached in node status ([#112303](https://github.com/kubernetes/kubernetes/pull/112303), [@xing-yang](https://github.com/xing-yang)) [SIG Apps] 1212 - Kube-apiserver: redirect responses are no longer returned from backends by default. Set `--aggregator-reject-forwarding-redirect=false` to continue forwarding redirect responses. ([#112358](https://github.com/kubernetes/kubernetes/pull/112358), [@enj](https://github.com/enj)) [SIG API Machinery] 1213 - UserName check for 'ContainerAdministrator' is now case-insensitive if runAsNonRoot is set to true on Windows. ([#112212](https://github.com/kubernetes/kubernetes/pull/112212), [@PushkarJ](https://github.com/PushkarJ)) [SIG Node, Testing and Windows] 1214 1215 ## Dependencies 1216 1217 ### Added 1218 _Nothing has changed._ 1219 1220 ### Changed 1221 _Nothing has changed._ 1222 1223 ### Removed 1224 _Nothing has changed._ 1225 1226 1227 1228 # v1.23.10 1229 1230 1231 ## Downloads for v1.23.10 1232 1233 1234 1235 ### Source Code 1236 1237 filename | sha512 hash 1238 -------- | ----------- 1239 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes.tar.gz) | 0e986db4a54b5d07773b17fae599680bf221c87d3c7ec185feba7aa23fe990f999af6ea28d8f5631b30f797104cbca8ac9a15bc029452d48b2eb9c6536c8abb3 1240 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-src.tar.gz) | 4e44687dc3594ede040199eea620ca330c41d323c70a69e1cf0d4b9ead28afca5b477d55e89eaf9ee05462bdda1aecb6c2382e2fdc0d156c1177d5f703e7c2ad 1241 1242 ### Client Binaries 1243 1244 filename | sha512 hash 1245 -------- | ----------- 1246 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-darwin-amd64.tar.gz) | 90d6164c0ae119eb187c7d878ea41fc825f3d9cdf466bba3e0758613fce67736131f28fc16ca9c26754b4d117c5ffa436e144b59df84da753b6066c67f92c453 1247 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-darwin-arm64.tar.gz) | 72fb1c734a6f0392ac98db78b365c0816fdf3c995fbfce35bd46034959c8cb5939b09affdcbc8e57866f754866456646c84dccc7a098909bfd57ec559329e64c 1248 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-linux-386.tar.gz) | 59dcae5a3cdf9e240017cba82ec07fa97ad5fbd8a17d09388aabc38529027fb4c9c67c55741157cc888e514929bcd5277fda5fe7eedfe7d63c3374b62dcb0e0d 1249 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-linux-amd64.tar.gz) | fae98e5e61dbf869428d14e05a2daa60d221a41688faf016157c5a09b400ed2a4a49c90681fea0bbacc75e0e158f75da62d026a6fedd37f9e52d56e3112f25c9 1250 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-linux-arm.tar.gz) | 681728f0ab837b1fb679e881fb0c9e728dc93bdfff9dbe3e333234166192d79b907a991d21e66a25ac56b16bc446332cb4bdc9891bb5334826f07c159617ab49 1251 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-linux-arm64.tar.gz) | 58339517a0a89204f357354173df493bea41149a100dd91f73f290f19ad8903b2e7733bf1c2acd21be9266b50a55f1dd0fc039af1f42ef98dc10ce0f86185ab3 1252 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-linux-ppc64le.tar.gz) | a3c703613749e61f4bb618fce2091fc5c10eb85fc4507838ca0c79765139948cfbc46ee60a2218d95863e0b2ff3369650a4ce0a59fdfd548a921fe6a3cea54f4 1253 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-linux-s390x.tar.gz) | 8f4d343205b3d2b0ba61f1639e76edf7934da3c7316095af510c9adc9a3a623065b4a62da4bbfe6ea95809020143751cf21ba774ba6b5ece4dd2de11efeefb70 1254 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-windows-386.tar.gz) | 292f13a099c04dd74af026f67a13d5f7ff85f215e2b5fbb784f07e2408ddf2f0041cac4205043415a5e8fdf4183e9589c8c68bdef90e426df0747f743fb3123b 1255 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-windows-amd64.tar.gz) | f89a6ecef52164f5f77274ea4240cb5000a9a3b200af85187d7194850ff2e5b8fc89fd41ad4311452ed80c8785b2a5f8cafe3c3de727eb68aa1a0b688f198647 1256 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-client-windows-arm64.tar.gz) | 16893e512a991180ca8e0d7c052e74b5e35cffb3a37ad92623ff376e2fc9d4b77edcdb6d7fb147c4de00a7b419ec04043aeb4f33a1e7114179885494c36c2948 1257 1258 ### Server Binaries 1259 1260 filename | sha512 hash 1261 -------- | ----------- 1262 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-server-linux-amd64.tar.gz) | d587c2106ef717e6c32d4480ff9cab768a4c58c40b65c8da61446b18aa071ac41a09d5d854e1577f3550b2df84fdf5685a3b9f5766acb9eaf51e2ae587af3b08 1263 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-server-linux-arm.tar.gz) | 244d57c42709c3f45aba508eb800ff228d091d474b8985689d455f916bbf15a93c081fdc6c1e1dfc7fac1905ea411cef085b88ebde41e0296d15146d2b49490b 1264 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-server-linux-arm64.tar.gz) | 4a512a1440d0cbdc2dd6a4e43e4aa6dd23835be576b6aa58b4cc9c5c502f607d3a7aebefbbedaf55c5778f3acf102d5c19a8822203e9638b3bd4752d23a46d38 1265 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-server-linux-ppc64le.tar.gz) | 1198ab579e87585f335c310fae173d344897fdc0a503c560612d7961b0cb737cced2a1a9408b144b15eeddbb37298dc997a75c984242e5b97b7f58009d0fc71a 1266 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-server-linux-s390x.tar.gz) | 3f655435f01106d64de55a2c688e086456ec2265c6719ab0c80064c49666c9b27b7664e69d76df32540873f78480d6f8751566c018d06c4018aa08f51c9e15c3 1267 1268 ### Node Binaries 1269 1270 filename | sha512 hash 1271 -------- | ----------- 1272 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-node-linux-amd64.tar.gz) | fc0a7e46e0b3611f7c3e3d50af377ab24b154590c50d75bf17572df71c5fb5fa2de50db246dba114f0714e8f0b5e7da70f01065bda6a9be2ada958af61d9c0a9 1273 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-node-linux-arm.tar.gz) | 6b00d621d862899aa858ee838b3767bf094b36ff44cd9f414b5d429a40f26f410942f4325652908fd92dd1117208a1163d246a309a6b3344750f6dece847ca9e 1274 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-node-linux-arm64.tar.gz) | 4fa47526aab15142e88d3954d68299583427511154076bcc849826671185621b9f2ddc9fdd31f958f5a257234d9def2f0795497cf4fdfaf5b6901204f1044862 1275 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-node-linux-ppc64le.tar.gz) | 56522b8eecdc8a85e337dab05957277fe1f086e7029d649ce234423d014ad48557d4aba160fc48f0df72c108a43b0862f7fea24ebd7e80a643a2e4a041181de1 1276 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-node-linux-s390x.tar.gz) | 3bded271505dc71328ba9e16917929e0b83c9a89615fc4683bf5dde565a9e4c5f54e85618bfb70457ecf3218317754efe2d599cc46efb9f2ffa6ea355c6336ec 1277 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.10/kubernetes-node-windows-amd64.tar.gz) | 5408c4da6e6866cc8a962b43e1f91981d7b95e530c3783d4f9849b9f848237569f061e8739f7cc134e6e5807627cc46e52c9a38ce9908eeacf319d09df7b738b 1278 1279 ### Container Images 1280 1281 All container images are available as manifest lists and support the described 1282 architectures. It is also possible to pull a specific architecture directly by 1283 adding the "-$ARCH" suffix to the container image name. 1284 1285 name | architectures 1286 ---- | ------------- 1287 [k8s.gcr.io/conformance:v1.23.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1288 [k8s.gcr.io/kube-apiserver:v1.23.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1289 [k8s.gcr.io/kube-controller-manager:v1.23.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1290 [k8s.gcr.io/kube-proxy:v1.23.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1291 [k8s.gcr.io/kube-scheduler:v1.23.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1292 1293 ## Changelog since v1.23.9 1294 1295 ## Changes by Kind 1296 1297 ### Feature 1298 1299 - Kubernetes is now built with Golang 1.17.12 ([#111465](https://github.com/kubernetes/kubernetes/pull/111465), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1300 - Kubernetes is now built with Golang 1.17.13 ([#111640](https://github.com/kubernetes/kubernetes/pull/111640), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1301 1302 ### Bug or Regression 1303 1304 - Fix JobTrackingWithFinalizers when a pod succeeds after the job is considered failed, which led to API conflicts that blocked finishing the job. ([#111665](https://github.com/kubernetes/kubernetes/pull/111665), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps and Testing] 1305 - Fix memory leak in the job controller related to JobTrackingWithFinalizers ([#111723](https://github.com/kubernetes/kubernetes/pull/111723), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps] 1306 - Fix memory leak on kube-scheduler preemption ([#111804](https://github.com/kubernetes/kubernetes/pull/111804), [@amewayne](https://github.com/amewayne)) [SIG Scheduling] 1307 - Fixing issue on Windows nodes where HostProcess containers may not be created as expected. ([#110965](https://github.com/kubernetes/kubernetes/pull/110965), [@marosset](https://github.com/marosset)) [SIG Node and Windows] 1308 - If the parent directory of the file specified in the `--audit-log-path` argument does not exist, Kubernetes now creates it. ([#111226](https://github.com/kubernetes/kubernetes/pull/111226), [@vpnachev](https://github.com/vpnachev)) [SIG Auth] 1309 - Reduce API server memory when many CRDs are loaded by sharing a single etcd3 client logger across all clients ([#111649](https://github.com/kubernetes/kubernetes/pull/111649), [@negz](https://github.com/negz)) [SIG API Machinery] 1310 - Updating kubelet permissions check for Windows nodes to see if process is elevated instead of checking if process owner is in Administrators group ([#111079](https://github.com/kubernetes/kubernetes/pull/111079), [@marosset](https://github.com/marosset)) [SIG Node and Windows] 1311 1312 ## Dependencies 1313 1314 ### Added 1315 _Nothing has changed._ 1316 1317 ### Changed 1318 _Nothing has changed._ 1319 1320 ### Removed 1321 _Nothing has changed._ 1322 1323 1324 1325 # v1.23.9 1326 1327 1328 ## Downloads for v1.23.9 1329 1330 1331 1332 ### Source Code 1333 1334 filename | sha512 hash 1335 -------- | ----------- 1336 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes.tar.gz) | 78d2929a7582e3b875bf00f921c580de42538aa05dd27c9cc16b64109c504aa11961408e07363d77b8ea0177ff17ebc4b37dc729dff32dddb713e864425d7494 1337 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-src.tar.gz) | 91e861a195735695a41035a694863b0483b7f4f7956b719c07ee0b31dd1a2f16b3e5e91643c0e35ad3f5ddedadabc307a728ede03ca94c901698353c65622431 1338 1339 ### Client Binaries 1340 1341 filename | sha512 hash 1342 -------- | ----------- 1343 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-darwin-amd64.tar.gz) | 1ef3d70708efc694932ba94b860044dcb0ad3e1513f25f88140f8391faa142a2160b35e479feee5612d6ff3f156dae8d9fd4b23e3f35471fe4932fb64ca8a188 1344 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-darwin-arm64.tar.gz) | ad087e869b0b9b79fb8a0f31ecb4145ee7fac338ad73dd3f64c904518ef04bc0e8a7f3b7ef403c1f3633980b368e2c9739dc7797ce2aa99df47d58766f169ee2 1345 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-linux-386.tar.gz) | c8c1111988d918f385f97ba1563369d33fef8b52204f621919e61980be0e6ee79242face11b46841f17dac8a0d5164bcf54de3668655debd90b831be0b7033df 1346 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-linux-amd64.tar.gz) | 3128800d337cc9be19494bcc724b9ea112444be6194136ef0e49b969394522533262a3f92554cf3543ae9b1e0bf063de7694f73ba230154bf2e59b74599f40ee 1347 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-linux-arm.tar.gz) | b5e125472055b35344b2ccfe34ecef545d8e3501a150479a1adb9c9218a96011de3ef36c9bfbb352ded41fdbcbd460bf4c600648fb586fec3bf9685eaac405b8 1348 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-linux-arm64.tar.gz) | 0bdde0bca2208de991ff3196ca73886f7129cd8037608ca8079973fe73d2742adb162fb693db6f30e0f4e4d15f7591e19e95ac1ce70d02cae77ded8e746f7b35 1349 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-linux-ppc64le.tar.gz) | 325a42eefc77305015ae23d61185ab728055aa622576c79ef5264120d0c00e118a2fe9c1bcbcc3250ee0088f380b9ea7f81755331ad552fe16e841fee55072e9 1350 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-linux-s390x.tar.gz) | 7586d553bee0a49226e973f0851b8f03394154849904e4b88f632d80924906c02321d78ba3ddfbd2be1c6328b7675a568fbc8ca8973bacfb6d11b6252d996479 1351 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-windows-386.tar.gz) | 68eb204d5bb4411d63c87838088db05d97349c10e6fae7f2e9f55ee934a347246f305045cd756195f20702937ab9c5082adaa84264f72841d81dbd8cbbcef6c2 1352 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-windows-amd64.tar.gz) | e21c10e5fe4f1b39f3c69e5c48030661ffc8e5fdfb20c8274ea61ad4adb4bccca557fa403a6250aa5ce17daeec11c330f9f29153748f5114b7275c2e87e10693 1353 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-client-windows-arm64.tar.gz) | 142a19090b3a169fc43982bd8b9a5d1d67855e27fe9ce2a9e863d83ba948d2e880c3b2ee3561a5d821f27dbdbc2b15af3eb2152d3244f1dff2829cf83b48e793 1354 1355 ### Server Binaries 1356 1357 filename | sha512 hash 1358 -------- | ----------- 1359 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-server-linux-amd64.tar.gz) | 90d5ac43e6e2794ade95b74328f68d1b87a6034b3ccfbb73ca8fadf14b2a98fd1700fe2790b4c963375516972505774e30c13a7cd4f10099f80dc4dafd357c0c 1360 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-server-linux-arm.tar.gz) | 5dad2818d5457572afd0a8b6c74004d9c36c94614601c237c72585eca214330fb3aefdf3cb778c17cf0fa3a062f6d9416b11ae32229479ed0ae603c148a3f28b 1361 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-server-linux-arm64.tar.gz) | 2cb1ae6078250a26dd405ad98d9440cc390f87b56543497299c0d51338cb563c38c57308fbea3e96f8c2ef56102d48b8bdd4ae6c22093ae884edebe14b0f766c 1362 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-server-linux-ppc64le.tar.gz) | f1905fa13f308aabe453a35bdc2d194d6a55e5db221e61ddc3436a628d850f708da19240801210ae4804c8786d82d3005d0f252dc969bedeccfc5141a409fb68 1363 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-server-linux-s390x.tar.gz) | e05f415e64424fb5c2ce17dc6008c9340e8a69964cc397a542286ed2a5970c19d415cfa13b2122958411b24893cbb3f817a627a1ac7f79fefa39ee0de204fed6 1364 1365 ### Node Binaries 1366 1367 filename | sha512 hash 1368 -------- | ----------- 1369 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-node-linux-amd64.tar.gz) | 44140e5ebe0441e4cf6a155250b08c6e321cd32208ac672ff46af97c01b40de42142ca019a892453f024db7894acb77e0421c3fc6f6058243d6c0ae55d212cd3 1370 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-node-linux-arm.tar.gz) | 797905b2ae3b07934ef6a3c546e1343d82ad3f45555d9963a1c4997c9bfaca79226b6825db8b36078289ec350864d4759dbe1e84548ebdc1a91439c9f0e4fdc1 1371 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-node-linux-arm64.tar.gz) | 4e9487951ed430e307686500ff44d371aac184d02b453495f6b643f24943d347739a9441322e94173afdb801ca47954c99d8e2744c7bbc6707b79a9733fb3903 1372 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-node-linux-ppc64le.tar.gz) | a1d74ef1d83aa1b095537a729acc61995ca1b18467febcc5ccf25399579d69aea05b0bdcab3c97e04a172edc61f54fca2ccebfbe9461a72079fbb800095375e6 1373 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-node-linux-s390x.tar.gz) | 4b4c5441f5d3d9ac7347a4da2db5e453986a2e0cb53cd17b77dfca57edb543ec6a96b7a9ca2e3732063dffe84cb1873c883dcc0562db336b367f8c902ee7f9b5 1374 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.9/kubernetes-node-windows-amd64.tar.gz) | 4710f1b88761e735f59ca4828c4e60fb9a125a86dca826355d10c6c14e9d195a300693e1a26c86b75fd9790ccf4fa803bb77969f20a64e096037b7a3dbb02294 1375 1376 ### Container Images 1377 1378 All container images are available as manifest lists and support the described 1379 architectures. It is also possible to pull a specific architecture directly by 1380 adding the "-$ARCH" suffix to the container image name. 1381 1382 name | architectures 1383 ---- | ------------- 1384 [k8s.gcr.io/conformance:v1.23.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1385 [k8s.gcr.io/kube-apiserver:v1.23.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1386 [k8s.gcr.io/kube-controller-manager:v1.23.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1387 [k8s.gcr.io/kube-proxy:v1.23.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1388 [k8s.gcr.io/kube-scheduler:v1.23.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1389 1390 ## Changelog since v1.23.8 1391 1392 ## Changes by Kind 1393 1394 ### Bug or Regression 1395 1396 - Fix a bug that caused the wrong result length when using --chunk-size and --selector together ([#110757](https://github.com/kubernetes/kubernetes/pull/110757), [@Abirdcfly](https://github.com/Abirdcfly)) [SIG API Machinery and Testing] 1397 - Fix bug that prevented the job controller from enforcing activeDeadlineSeconds when set ([#110545](https://github.com/kubernetes/kubernetes/pull/110545), [@harshanarayana](https://github.com/harshanarayana)) [SIG Apps] 1398 - Fix image pulling failure when IMDS is unavailable in kubelet startup ([#110523](https://github.com/kubernetes/kubernetes/pull/110523), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 1399 - Fix printing resources with int64 fields ([#110602](https://github.com/kubernetes/kubernetes/pull/110602), [@sanchezl](https://github.com/sanchezl)) [SIG API Machinery] 1400 - Fixed a regression introduced in 1.23.0 where Azure load balancers were not kept up to date with the state of cluster nodes. In particular, nodes that are not in the ready state and are not newly created (i.e. not having the `node.cloudprovider.kubernetes.io/uninitialized` taint) now get removed from Azure load balancers. ([#109932](https://github.com/kubernetes/kubernetes/pull/109932), [@ricky-rav](https://github.com/ricky-rav)) [SIG Cloud Provider] 1401 - Fixed potential scheduler crash when scheduling with unsatisfied nodes in PodTopologySpread. ([#110853](https://github.com/kubernetes/kubernetes/pull/110853), [@kerthcet](https://github.com/kerthcet)) [SIG Scheduling] 1402 - Kubeadm: fix the bug that configurable KubernetesVersion not respected during kubeadm join ([#111022](https://github.com/kubernetes/kubernetes/pull/111022), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] 1403 - Reduced time taken to sync proxy rules on Windows kube-proxy with kernelspace mode ([#110702](https://github.com/kubernetes/kubernetes/pull/110702), [@daschott](https://github.com/daschott)) [SIG Network and Windows] 1404 - Updated cAdvisor to v0.43.1 to pick up a kubelet fix where network metrics can be missing in some cases when used with containerd ([#111013](https://github.com/kubernetes/kubernetes/pull/111013), [@bobbypage](https://github.com/bobbypage)) [SIG Node] 1405 1406 ## Dependencies 1407 1408 ### Added 1409 _Nothing has changed._ 1410 1411 ### Changed 1412 - github.com/google/cadvisor: [v0.43.0 → v0.43.1](https://github.com/google/cadvisor/compare/v0.43.0...v0.43.1) 1413 1414 ### Removed 1415 _Nothing has changed._ 1416 1417 1418 1419 # v1.23.8 1420 1421 1422 ## Downloads for v1.23.8 1423 1424 1425 1426 ### Source Code 1427 1428 filename | sha512 hash 1429 -------- | ----------- 1430 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes.tar.gz) | de69d27776c066cdc450b65d075fff9efe39175823aa3cd65cf8e431ba397c38add39ef53fc4d21b74c900d9bb5ed055a0f05d7a25a429c2f1f3967c2c5ff01c 1431 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-src.tar.gz) | 25386d6c7c0fc9829bc178c8a24b3d0fb65895495c0c3f25e6a3072728ee39e4b55855e1587a6dbf2fe75bd8a192d908dde7843c118f00fd71c9f76b14a9180a 1432 1433 ### Client Binaries 1434 1435 filename | sha512 hash 1436 -------- | ----------- 1437 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-darwin-amd64.tar.gz) | e4a95fd15f417b51baf73f096a940adbe4426609f0fde2077c93b50e7b0cd5da8086e753c47112bc71575b00f8efaa030f92abc5f100a7113b9375ba4b9f58e9 1438 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-darwin-arm64.tar.gz) | 4296920ddfea2fee14b680a231a72688771030387112f51f502c52662b3c72f51d254d579ca0f24ed6d6327056f1b047e52d43f2ff2bb6bf4267b3b30f1cbdbc 1439 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-linux-386.tar.gz) | d600fb12e7fa00374875efb6d2ede65790cb64b149bca6db2eecdba1fc8845a0a46126041cef3bb3489180ce2825ed116257098f6009912635f1bb6560977c21 1440 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-linux-amd64.tar.gz) | 071de5987abc6295c3f9843a31b3fc0bea106f1ca3498bf089f867c3914a20a8fc563fc0763640c589691fbd9ab7dc4280d8f8de2b2904bde13f8c4f059580c9 1441 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-linux-arm.tar.gz) | 4677affdf764a5983ab64af3876c120919ce319e0eb67f922439a93f13a3f62de8b81c652f9d1947572c1f051f49a54157717d6a1416b49cde7f76c49b8e342e 1442 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-linux-arm64.tar.gz) | 3ac6eb2dd77a71ea608777429a8efa0c26aa3bf835f125a114c3d2312337a00c5b96eaf32edaef319c055be023d9862ff6f7fc06c0056d4b2491eaafe34315b0 1443 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-linux-ppc64le.tar.gz) | 233efa4ace4c9ac36b9eae75c32d477adc01c0690ecc069b4daecbbe2aabb50514255ef659bc9a2ffd2ceeeadd33965aa743299bf02814d200b11dfd2061d170 1444 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-linux-s390x.tar.gz) | 841d6a53cae230bcdf58e65ab6d780737c41c99f0144443905caad62b508b94f738640e2fb7a43ea39bb82387ba537389fb86d555572de1eb6adcf8d80fe31b4 1445 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-windows-386.tar.gz) | eefcb9290ac80c6d0e50fa77d4fabd8b71949412c37272ad33468bc31251d031b877cb26e17ee56d51616347331f5f9d17fa170a44d1f0f9530be9c3605981dc 1446 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-windows-amd64.tar.gz) | 30b82098d3dc3fbc045174dddc273ba38ade9a64e6fb5a4a70646343752ed882f7342bc78c70b6af02fe9b42a20ef84763ea89418aa98618a19c39945aa34eb3 1447 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-client-windows-arm64.tar.gz) | 76c7f0e5ae9158d52a3bf76e638fc750bda8a4f207a9cfe3d99f2f1035b85d4feb8443de01d2daae2bf2414814286929b523341d896c144b6383b7a98b482cf1 1448 1449 ### Server Binaries 1450 1451 filename | sha512 hash 1452 -------- | ----------- 1453 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-server-linux-amd64.tar.gz) | ca0c4023c1bbeb1aafafb073348e42a016f6793b64a84b33f9619971950ee0698cc6f008418c707cf5126ca82bff936508e63e02ecf89f38997d2aeb2f5facf0 1454 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-server-linux-arm.tar.gz) | e33be64370b1418aa3ef935f1a01e4aff3285c0f3dbd8b0af1be60af5bb245f291671b557cda6c41e4af82da49b42bbe1562e568011099195e24f8add036227d 1455 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-server-linux-arm64.tar.gz) | 86350da9262bdad99d997bd4a6f1f20067a77e2a8d0b4359d7681dfe0a272b42fa30bc4de47920aa8aadcfb5ab0d9b626c8fe3c268db0ff9c35d0f32f94e8cb0 1456 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-server-linux-ppc64le.tar.gz) | d0954bd675eae099195f8986a9c2ada9eadf804794f8bd91371c53423173398457987ef9ca73664f097f8301486f6416f31a7540bbbcbc844fe3ca7fb58529ee 1457 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-server-linux-s390x.tar.gz) | 017d4026ff30bb10460d22571a592c50f6c9496b49acecb8a333174b33a8a98cd3f234d49546d45947d9af1c3ab5a1e337b42015968e9572b482bf7a4a58d181 1458 1459 ### Node Binaries 1460 1461 filename | sha512 hash 1462 -------- | ----------- 1463 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-node-linux-amd64.tar.gz) | 3003ba3faa6d175c2e0b898b3a07b3bb22e6509a9fc7948582d92009a648b83bbe329a8c2f843a466f94089fd6a26529c89abef8c769f406a55f39a09bc55017 1464 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-node-linux-arm.tar.gz) | 3938d4d33cbf021108ea1dc03534286c58ad68ee059027a239548cd2bce3708eb288f6893368014f885b20ebe26c4d556d27cf8fdf29bf478e5ed23bdbfe5325 1465 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-node-linux-arm64.tar.gz) | badf58767eae4a20f81200f609fd5966479b455a3bd9d3833d5cbb0e78c31e6d624c7301cc366b5b07da6299d2d5e0228db2aaf78c43e947a3a80a4db210965b 1466 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-node-linux-ppc64le.tar.gz) | 301c234325e9a588315ff128469c49b64f471c6a40ce0567b745f8315bbe8847b8ba9e6e098c317bf20f0613e65e3d604afba12edcb84154e636427139344cc5 1467 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-node-linux-s390x.tar.gz) | f862838e1f7294de4b58359f0ff4e4644ff6229f83bd615f35aa5b625a1754a4c6e538fb2f911c13ed434fd81d2e2feefdbd4fd0148149ea24d3f6df8c01a7ec 1468 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.8/kubernetes-node-windows-amd64.tar.gz) | cecb68bad4aea5790afc72c895d20d93bac53985437dbddb0917c7b9b8f876831758a66595c70fd8c142140a2eecc8d2fc2a52885ca4d062ea34586a2871f133 1469 1470 ### Container Images 1471 1472 All container images are available as manifest lists and support the described 1473 architectures. It is also possible to pull a specific architecture directly by 1474 adding the "-$ARCH" suffix to the container image name. 1475 1476 name | architectures 1477 ---- | ------------- 1478 [k8s.gcr.io/conformance:v1.23.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1479 [k8s.gcr.io/kube-apiserver:v1.23.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1480 [k8s.gcr.io/kube-controller-manager:v1.23.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1481 [k8s.gcr.io/kube-proxy:v1.23.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1482 [k8s.gcr.io/kube-scheduler:v1.23.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1483 1484 ## Changelog since v1.23.7 1485 1486 ## Changes by Kind 1487 1488 ### Feature 1489 1490 - Kubernetes is now built with Golang 1.17.11 ([#110423](https://github.com/kubernetes/kubernetes/pull/110423), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1491 1492 ### Bug or Regression 1493 1494 - EndpointSlices marked for deletion are now ignored during reconciliation. ([#110483](https://github.com/kubernetes/kubernetes/pull/110483), [@aryan9600](https://github.com/aryan9600)) [SIG Apps and Network] 1495 - Fixed a kubelet issue that could result in invalid pod status updates to be sent to the api-server where pods would be reported in a terminal phase but also report a ready condition of true in some cases. ([#110480](https://github.com/kubernetes/kubernetes/pull/110480), [@bobbypage](https://github.com/bobbypage)) [SIG Node and Testing] 1496 - Pods will now post their readiness during termination. ([#110417](https://github.com/kubernetes/kubernetes/pull/110417), [@aojea](https://github.com/aojea)) [SIG Network, Node and Testing] 1497 - The pod phase lifecycle guarantees that terminal Pods, those whose states are Unready or Succeeded, can not regress and will have all container stopped. Hence, terminal Pods will never be reachable and should not publish their IP addresses on the Endpoints or EndpointSlices, independently of the Service TolerateUnready option. ([#110259](https://github.com/kubernetes/kubernetes/pull/110259), [@robscott](https://github.com/robscott)) [SIG Apps, Network, Node and Testing] 1498 1499 ## Dependencies 1500 1501 ### Added 1502 _Nothing has changed._ 1503 1504 ### Changed 1505 _Nothing has changed._ 1506 1507 ### Removed 1508 _Nothing has changed._ 1509 1510 1511 1512 # v1.23.7 1513 1514 1515 ## Downloads for v1.23.7 1516 1517 1518 1519 ### Source Code 1520 1521 filename | sha512 hash 1522 -------- | ----------- 1523 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes.tar.gz) | 3dcd42ec4fb7fbc3b00dc18414bcab7438c22b8040e9d66998e844b289fba31b3b951800cc9e6d88722d1868c27e7dd3873c6d761d43c247570f83e8e6e5ac97 1524 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-src.tar.gz) | 80482eef8ea1c07c0c5e272d8b0fb56c22998d20f40c275d19d3d33197f4a7a1c4a71783f3cbb71ceb21c5ac990ee253d28888eae68e8dbfbee3c39d0f047af6 1525 1526 ### Client Binaries 1527 1528 filename | sha512 hash 1529 -------- | ----------- 1530 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-darwin-amd64.tar.gz) | 15ed16d69f01e585e5272e0284f7c9112eaee22d73afd781de40f8f6d40e71ecc46cd6951beb3a171b539c8b343ad695ed67ba67779b18d8c668a20ef07e1c35 1531 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-darwin-arm64.tar.gz) | 0f8eecfef713c2eea9793f36c90150c2d7dc444c69504f303f025c74b1f74a4d92fd61af379765c01eebdfb696812f98f256225d5cdbbfc34c26d0d9c4874ced 1532 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-linux-386.tar.gz) | 16a86a9f6bf6c28c541d8b73e6704693d31fed4a09afa16d8b5414fd091133e2746c5f477f6571eec9a7540be88a9ab18ce644e99e6c8ad6e79506e73b0a8211 1533 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-linux-amd64.tar.gz) | 971d2bb7fb2ac098829a91a5451687bf016f8cccda80fa605b19a55e99b5a411ff838d8ec25b5abd2e3984b6807c7ecb9dd1ba744279be65e45495443ecb1726 1534 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-linux-arm.tar.gz) | 6c003e494b26f7ff03653667db2abce5533817d733c95c64b5782374a4ca40eaa54682a5f7d0e6bb5043c3e43676fed868f6bcbb233b410bc999b276bf65a8e7 1535 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-linux-arm64.tar.gz) | 9a07bc8671a398c01aa31c5c3facc1469b708a7f388081c4e4110ee852af8a6d74cc700c925b3c867852e259477a232a992eda4c1211a9ac91233dc3de5f7ee1 1536 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-linux-ppc64le.tar.gz) | c60592be14d218799dd15360d004adbabd26baf73137f3ada526fc2d6e6b7e76dc8b2dcba939fc2c92d23f1a837fb5ae788ddc1ff06b5274f5aee503e27cefd4 1537 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-linux-s390x.tar.gz) | 6958d77d1f126a41ddeb4f5e7b5dea953c8174924165a5ded0c5b518349ae526cfcc869f8d9aa6f4cf078480877b89a9dd6a5b4c2f40b556e16f1618cf74e32b 1538 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-windows-386.tar.gz) | 5ec708da681544ee7c8729f5c3afc44c869c3be4b8fdaaed4668f13c2b9a40f8a138dd01b93595f84ec4b3982027344a3146b7aea1617f60ff752f8182b2f792 1539 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-windows-amd64.tar.gz) | 60e52a530a4dba6d79abe21c55a3214b070840dab97f6da193cbe4f41b0a5a0707378ae7d118a018d5aec16b8a149784064d37f1f29cad635b9c876e0df744d1 1540 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-client-windows-arm64.tar.gz) | d5d35a538ba553c92601bd487b309269b0cd8c26c108d6ea4d107326d31ca58c44292bf0fc12612a5a2357f64053cb042a09cddb6fea6aac6a4d4e62019fe53d 1541 1542 ### Server Binaries 1543 1544 filename | sha512 hash 1545 -------- | ----------- 1546 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-server-linux-amd64.tar.gz) | de3dccf8a7e47542094528f6738b1d34073977c6031318bacb1656656860054e6cfe21280b0f314a1f1c73d393929341ad4866604ee73eff37af5087825a766b 1547 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-server-linux-arm.tar.gz) | 5ccdc82f5289bb7177b81e909c742a43f6dc8c5cc80abeb97808a7097ab08019eea0ae5c959f7f938457cc684eb1ed8c90e5f436a1b8e85d91a7dbe9a34e331b 1548 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-server-linux-arm64.tar.gz) | a463d637fe8becb4e1b5b99b1cec44538ff31a386c7d331ba7791904773f931630840657f723212af65e519a7830b47bf50063616ee8c906212a6751bf426333 1549 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-server-linux-ppc64le.tar.gz) | 09ce40a6a684e0060ba41a9dd5eeb6da9aad9098f284b4b082ac386fe12fdb55514c4987455cb6aaf80acb74c637b77828ce7909da82bcf3f2afc094f3864bee 1550 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-server-linux-s390x.tar.gz) | 3d3f904f935915aa12b36ac80c43275250c7a4b9a169aa641e225592bbdbc9da28c923056458211e5d3aec6f7344477d5fa9a9239d59f47599836082926f5de6 1551 1552 ### Node Binaries 1553 1554 filename | sha512 hash 1555 -------- | ----------- 1556 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-node-linux-amd64.tar.gz) | 9dd2ec04902bb6dc0eb0a75735c8f2958ab7dc106487ac962564e6fc514c90581ac32e15e8fc658c598f0b6c3d199a938c4824bda23157f379510a8cdecc4a7f 1557 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-node-linux-arm.tar.gz) | d73d8cee8bac5b977ec0b11917a4d4eb8d5d33c8a2548210a806a3882e449d242504934ec199bcac80c31c53542f883f69d48c599eab3fe2cd2e3d95760ec966 1558 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-node-linux-arm64.tar.gz) | 4498a1b60813f0e1461c00d5cb2351a522f3e7c9d9d81ffcc99bf925378885aa5dbb64fae00dda3afdbe2f9517eb30ea28cc18c11158a9a46f69c0bc3244f2c2 1559 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-node-linux-ppc64le.tar.gz) | 50353816f8772d830d8ae05e1a535553fbcc2d552d00ba822b777c17b8492ba4ef5acd03b1d9f56583064893f173e3cf7b67faa4bdd77c11261a21c32bac8e4a 1560 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-node-linux-s390x.tar.gz) | c2c4b35a42ae84d498c5e27175bcbc641e74aa42056df2f36c6bdb35f6330b5aca26c2d5b8656750f13b75d02c092d0f0ac5d4ed287d13033f696f243af9ca39 1561 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.7/kubernetes-node-windows-amd64.tar.gz) | dde3429d945c4b8c22a551e8663678e76d7a1136b9f9012ead3af534582f738092b0d5e629a983ad446d0396db143188a440edf471ee8c102a331006fcd00ab8 1562 1563 ### Container Images 1564 1565 All container images are available as manifest lists and support the described 1566 architectures. It is also possible to pull a specific architecture directly by 1567 adding the "-$ARCH" suffix to the container image name. 1568 1569 name | architectures 1570 ---- | ------------- 1571 [k8s.gcr.io/conformance:v1.23.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1572 [k8s.gcr.io/kube-apiserver:v1.23.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1573 [k8s.gcr.io/kube-controller-manager:v1.23.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1574 [k8s.gcr.io/kube-proxy:v1.23.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1575 [k8s.gcr.io/kube-scheduler:v1.23.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1576 1577 ## Changelog since v1.23.6 1578 1579 ## Changes by Kind 1580 1581 ### API Change 1582 1583 - Sets JobTrackingWithFinalizers, beta feature, as disabled by default, due to unresolved bug https://github.com/kubernetes/kubernetes/issues/109485 ([#109491](https://github.com/kubernetes/kubernetes/pull/109491), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps, Auth, CLI, Network, Node, Scheduling, Storage and Testing] 1584 1585 ### Feature 1586 1587 - Kubernetes is now built with Golang 1.17.10 ([#110045](https://github.com/kubernetes/kubernetes/pull/110045), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1588 1589 ### Failing Test 1590 1591 - Allow KUBE_TEST_REPO_LIST to be a remote url ([#109512](https://github.com/kubernetes/kubernetes/pull/109512), [@eddiezane](https://github.com/eddiezane)) [SIG Cloud Provider and Testing] 1592 1593 ### Bug or Regression 1594 1595 - Correct event registration for multiple scheduler plugins; this fixes a potential significant delay in re-queueing unschedulable pods. ([#109446](https://github.com/kubernetes/kubernetes/pull/109446), [@ahg-g](https://github.com/ahg-g)) [SIG Scheduling and Testing] 1596 - Existing InTree AzureFile PVs which don't have a secret namespace defined will now work properly after enabling CSI migration - the namespace will be obtained from ClaimRef. ([#108000](https://github.com/kubernetes/kubernetes/pull/108000), [@RomanBednar](https://github.com/RomanBednar)) [SIG Cloud Provider and Storage] 1597 - Fix JobTrackingWithFinalizers that: 1598 - was declaring a job finished before counting all the created pods in the status 1599 - was leaving pods with finalizers, blocking pod and job deletions 1600 1601 JobTrackingWithFinalizers is still disabled by default. ([#109486](https://github.com/kubernetes/kubernetes/pull/109486), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps and Testing] 1602 - Fix a bug that out-of-tree plugin is misplaced when using scheduler v1beta3 config ([#108890](https://github.com/kubernetes/kubernetes/pull/108890), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling] 1603 - Fix kubectl completion zsh to use any command name rather than hardcoded kubectl ([#109235](https://github.com/kubernetes/kubernetes/pull/109235), [@soltysh](https://github.com/soltysh)) [SIG CLI] 1604 - Kubeadm: add the flag "--experimental-initial-corrupt-check" to etcd static Pod manifests to ensure etcd member data consistency ([#109075](https://github.com/kubernetes/kubernetes/pull/109075), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle] 1605 1606 ## Dependencies 1607 1608 ### Added 1609 _Nothing has changed._ 1610 1611 ### Changed 1612 _Nothing has changed._ 1613 1614 ### Removed 1615 _Nothing has changed._ 1616 1617 1618 1619 # v1.23.6 1620 1621 1622 ## Downloads for v1.23.6 1623 1624 1625 1626 ### Source Code 1627 1628 filename | sha512 hash 1629 -------- | ----------- 1630 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes.tar.gz) | 6cf7b3b00ab86a74cbaff599b121189ae73f7506b5041ee5351def4c2b9ee689f0fe23ff5a6ee682d994d3fdc47e54ba32640fd915a384cd33dfe7213f0e001b 1631 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-src.tar.gz) | 3a1acc0889add88f72bdfdababb54da8ef61e2d82176e2b102a5b73c3f3110123a9732bc5a8c657c75ce1bf379d1b13b3cc1941042c9f67644f39cf55891d24b 1632 1633 ### Client Binaries 1634 1635 filename | sha512 hash 1636 -------- | ----------- 1637 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-darwin-amd64.tar.gz) | 96faea4fb5df5c34b4f9affcf1612a70370af718d87587ed1f1e28d9df8c57e469d5902c09444413fed5f5dd578aff88a9efec9c2df201b1cbc461d701469c7b 1638 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-darwin-arm64.tar.gz) | c30428e83089d38f47fd39a583324b122fc19a9f39ee4454f218f61352a4f7638e94f9bc1d223ce76732115999434ce039e4762df954629126460566238c0027 1639 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-linux-386.tar.gz) | a0da9c6f41c0164c81b7e805533020221969fb68f0816c58e1e982a2a744350eb552aaec2c5e4a9410568dbb6b72225c901140cc308f132843cc53a058e6f47f 1640 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-linux-amd64.tar.gz) | 626620f7943575f4568b625b27e4df5e43073cf395f5666b7e44424a5964f913a6453e8fdd13990a52ca3187eb15c0e73a5f8d6ca2f297ca842afa0d111a7254 1641 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-linux-arm.tar.gz) | d8ab4acff263761d89ca95cfc207d17d51fbc8c6b79f1cfa9782ea2afa15cad8341265b74a834b0bad808f24bc40bfeed1e12e4dd4a31cf82aaf103a5ce6c4a1 1642 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-linux-arm64.tar.gz) | 28006d52582842479f91094506696957a99a74fefaf3dd1e44ded910468f616ef918b40935027cd431026c9a8ac70dfb5adb57fd6f2ec96e717258d1326dbca6 1643 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-linux-ppc64le.tar.gz) | db8e85c87fb12beb12798e66a49eccd72f590eec1167be226960786276c8d31f9aed694d4a0e96efccdd29886b587f5e4ff6ccddaf354867dabba910ef7396f2 1644 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-linux-s390x.tar.gz) | 9e3e11398fb5f980517ef4eb9e732db6fb0301e820a50d141e6bf90775728c131c0f0eeae183c0b9314533089e7f44643abb0b55b1c2a1e38a7cea4f739acbae 1645 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-windows-386.tar.gz) | b0afd0e85826fc061978e753682926b61a7b4a8618a8e0033185f69774afc7e7bd0de459d7cfb2bdc42f34398959953a8e57752766b9c283f517d6c6ce1c6ce2 1646 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-windows-amd64.tar.gz) | 76c7564d4910a03836eb43d5c18692b69d54862e30dea11994f5c4656c8911c08f083a2f0b0ee1e4756a4dd659b1a549bdeece8aa571e4e2ca7beaffb8523c44 1647 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-client-windows-arm64.tar.gz) | 1f9974ebeba3b8f751c561a64cc51c7ea109fb595c5177dfae87e9786804b19302fa96d7bac90fc8d99c04ef48d7400a0b11636a0264034749c7931bd01cf687 1648 1649 ### Server Binaries 1650 1651 filename | sha512 hash 1652 -------- | ----------- 1653 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-server-linux-amd64.tar.gz) | a33fd6a01b9a10e00fd6cea33c0e621fc96c83ed434564ec3c36d1c2589d5466413b3c2993dacf5d4d40560ea3f99afff03e126b4707f1aa0816324d3595af24 1654 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-server-linux-arm.tar.gz) | d6d7a7d16538a3484aff2c25358db81f1bbeb01c8a41930712e086a407fe9099e0100b7e4981b86e85824478aa1091d71deab211dd87582fb2869190cb8bfbeb 1655 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-server-linux-arm64.tar.gz) | c5f57460c375f7eaae7fcb14781c8131a1ae20392319648191daddd2397246831f1c5d2b8d3e4d859df5e7c17e23056279553a4b3e72835a9fe08a55da500140 1656 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-server-linux-ppc64le.tar.gz) | 219bb24fc0a483e325bde0b982ba10a561588dc39b48eb5fd0e1cc31d1b6c2cbb6217a424a2eb5c64f4210fc3d9f692308171341b075a6b07c10c7932037c0ae 1657 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-server-linux-s390x.tar.gz) | 90d0cca8028df8bfa260f4c9abba70747a5aeedcd59a92459bb5e94c2f0caefa39c68f11f38f916994d13fb5553134faa92ed735ed329f588b2c10874f95202b 1658 1659 ### Node Binaries 1660 1661 filename | sha512 hash 1662 -------- | ----------- 1663 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-node-linux-amd64.tar.gz) | 8083afd4dd05659e364d3592f71b0694f9c3d4c20d2260efbb2814ff8b914be3ced6d4690f9cf546522168b1ad6aab2ea70d9092cff0c7207db61514294b452d 1664 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-node-linux-arm.tar.gz) | 1c46b5cf9a2baf9d4c117a57efa64447701d1e55e249ab8a367008630e05036607fa0afc6c0e3eedec3efecd1a4d5c13316ac1034ae97e9d4971fba0f0c2e2ff 1665 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-node-linux-arm64.tar.gz) | 5e36c75eeede07c31ac29961460ee5cc32e6ffa7f53cf6f674e2087d11668f6bf27013daf72a4137e041f17c114d263f5c7b6c891ba084ecc76029f63bba1011 1666 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-node-linux-ppc64le.tar.gz) | c4eba116db85a94e5e647d8ca7f5d334a9aa8d42c27e10077eebce28e7948adb06d6cefcf10db8abd762960dcd7d4b6ab2c1caf7b079fc84bfef356cb822eb07 1667 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-node-linux-s390x.tar.gz) | 50905829928ac041d95243bac78879aa607d7a5bdadb92495e022f856e1f332204ec4e0cc2b9c848b1afddf1cb8b0fe249944537005a97967237581bc42e48b9 1668 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.6/kubernetes-node-windows-amd64.tar.gz) | ff71310b0453efb8ba364572eb5ade35a9826f3a6b1d5d215764b5e165054c3cd77e24b7f75a443c9ab577f39899212fea0fe849e3d0a8bbf786ac568f2c63be 1669 1670 ### Container Images 1671 1672 All container images are available as manifest lists and support the described 1673 architectures. It is also possible to pull a specific architecture directly by 1674 adding the "-$ARCH" suffix to the container image name. 1675 1676 name | architectures 1677 ---- | ------------- 1678 [k8s.gcr.io/conformance:v1.23.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1679 [k8s.gcr.io/kube-apiserver:v1.23.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1680 [k8s.gcr.io/kube-controller-manager:v1.23.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1681 [k8s.gcr.io/kube-proxy:v1.23.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1682 [k8s.gcr.io/kube-scheduler:v1.23.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1683 1684 ## Changelog since v1.23.5 1685 1686 ## Changes by Kind 1687 1688 ### API Change 1689 1690 - Omits alpha-level enums from the static openapi file captured in api/openapi-spec ([#109179](https://github.com/kubernetes/kubernetes/pull/109179), [@liggitt](https://github.com/liggitt)) [SIG Apps and Auth] 1691 1692 ### Feature 1693 1694 - Kubernetes is now built with Golang 1.17.9 ([#109462](https://github.com/kubernetes/kubernetes/pull/109462), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1695 1696 ### Bug or Regression 1697 1698 - Bug: client-go clientset was not defaulting the user agent, using the default golang agent for all the requests. ([#108791](https://github.com/kubernetes/kubernetes/pull/108791), [@aojea](https://github.com/aojea)) [SIG API Machinery and Instrumentation] 1699 - E2e tests wait for kube-root-ca.crt to be populated in namespaces for use with projected service account tokens, reducing delays starting those test pods and errors in the logs. ([#108860](https://github.com/kubernetes/kubernetes/pull/108860), [@eddiezane](https://github.com/eddiezane)) [SIG Testing] 1700 - Failure to start a container cannot accidentally result in the pod being considered "Succeeded" in the presence of deletion. ([#108882](https://github.com/kubernetes/kubernetes/pull/108882), [@rphillips](https://github.com/rphillips)) [SIG Node] 1701 - Fix indexer bug that resulted in incorrect index updates if number of index values for a given object was changing during update ([#109137](https://github.com/kubernetes/kubernetes/pull/109137), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery] 1702 - Fix the overestimated cost of delegated API requests in kube-apiserver API priority&fairness ([#109216](https://github.com/kubernetes/kubernetes/pull/109216), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery] 1703 - Fixed a regression that could incorrectly reject pods with OutOfCpu errors if they were rapidly scheduled after other pods were reported as complete in the API. The Kubelet now waits to report the phase of a pod as terminal in the API until all running containers are guaranteed to have stopped and no new containers can be started. Short-lived pods may take slightly longer (~1s) to report Succeeded or Failed after this change. ([#108723](https://github.com/kubernetes/kubernetes/pull/108723), [@bobbypage](https://github.com/bobbypage)) [SIG Apps, Node and Testing] 1704 1705 ## Dependencies 1706 1707 ### Added 1708 _Nothing has changed._ 1709 1710 ### Changed 1711 _Nothing has changed._ 1712 1713 ### Removed 1714 _Nothing has changed._ 1715 1716 1717 1718 # v1.23.5 1719 1720 1721 ## Downloads for v1.23.5 1722 1723 1724 1725 ### Source Code 1726 1727 filename | sha512 hash 1728 -------- | ----------- 1729 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes.tar.gz) | ced23c737deb52c9e84af8d2937b19d13822c05d588c29bde120a125c97276ea1f008c649e9a2117b6f026f5fc057afe0089d8ee1154c75e67e054e8ec73b847 1730 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-src.tar.gz) | 7fe3cf10bb534a26ee74fa1eedad0c98d874d8de0b8e2c91ed10ebf880e74968bd39d4b6e7170d694ea68bbdc775b3ca70ab78ec5f4f2d15bcb20d6e869f4b2c 1731 1732 ### Client Binaries 1733 1734 filename | sha512 hash 1735 -------- | ----------- 1736 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-darwin-amd64.tar.gz) | 0c5dfa5cd898aba6599273da7c16aa81f11d0e3a031d92820c98daabb2d2372c8a1dcaf2cb203d88ff172066239c4c783810356840a0b19603366f7760f41ae4 1737 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-darwin-arm64.tar.gz) | f41e58a9118378685ff42499d082f34bed72a23ab55c1e94cd6ec209aad8e5de943c9385d9b449bb70325b4a01f58f63f0784c43230a681bd13603ce034d85e0 1738 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-linux-386.tar.gz) | 795f9f68447b36ed912cad4b3f874780bbc28ef7dd34881927467ee42961563a8e6cb1d6d150f79a579ebfeaca2631bd99703625770b851e30dacfec5fbdde81 1739 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-linux-amd64.tar.gz) | 51f5679a0cb11a65f25c3479bbfdfd21c4d0acd8814d3cbaf5aaeea7682178a3820c3555b17ea6ee24470ac67ebfd0f78cc98513e5b526436494350be64bda69 1740 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-linux-arm.tar.gz) | 056d62df16a9725c7ae8b072cadb4b713bbd2230fff95d777721a951b6a1443c480920dd000039f1ef4da1df4ea8fabef096d7166a6dace5e6e6fe3b3e67da53 1741 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-linux-arm64.tar.gz) | a6b5b19a71e971cdfb0b4819add8f7ef3c24a99b6201b67f52ef6c65787a4d9008ba69b6f950bd162d05e7620c34971346a6b4b7cbeefe2ffe4a84a32661f208 1742 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-linux-ppc64le.tar.gz) | 969d000f87e991755f91f9b16c114c8606d342f669625111609c1991537e7085eef6c20a815c0c0890e2a405e015f9c5daacaab526b444fa5e0c075f0ac4d017 1743 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-linux-s390x.tar.gz) | 64a1bb89a47a37e7de1bc8835963d8ef3253c86306e5c4ef4d6b2609699fa5dce1d8611406c1d8bf22b3a7ec6194f61e526c8276381f79bad0fd173d6cdaf50f 1744 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-windows-386.tar.gz) | 381d1602538fd7926758fad59a64dba6fa560ecc48593cb55f7b3bcb494ac221dab84be1f9e945036a8cb6336c8a0ecdfc33c0e94f8613112dcda2e020e2646f 1745 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-windows-amd64.tar.gz) | a794fc29d9d2de0d5550a05dd7712b91dd39dc6c75bd9f291c25fb4acd3a5b6fcfaea07c768e291bd62b10c9b27d3d0f57af450ab489d872a3af7b89852c5878 1746 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-client-windows-arm64.tar.gz) | f4225d21f12a270019c7a96330f14159fe8a34b869370deeff3920b8dee3cd78e9173605033b9e138ccdef7304bbc39ac41ed582c0360b580a1b0cd67091e6b9 1747 1748 ### Server Binaries 1749 1750 filename | sha512 hash 1751 -------- | ----------- 1752 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-server-linux-amd64.tar.gz) | 063ad74fb1463ee7a7bf4fb746eef1e02980c170cfa89c93444bee0841a84133bdbe91035f6608ece15096fec3e0c9aa50ebdc2b15ce589d86e2f07d10a1d747 1753 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-server-linux-arm.tar.gz) | baa1a310236fc5baad609285fc4717913791cbad920dca17d8736e0052af9cad07ee454323e8d8c03cad70456bd69d79cececb7019b3f3a48978552bed3d2b68 1754 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-server-linux-arm64.tar.gz) | b3388b6da8fcbbaa30ac881f0f0dbf6ca501bd5fc52aca33174025fe6234af2872ab1a25a5a74049cfc627359a748c7cbc0f129ad3b800c4707d44f98ac69d52 1755 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-server-linux-ppc64le.tar.gz) | 1eab49c6ad3bc7f368b2756239f60b59cb946f6bf56974a5c62688f5bfc5175e2d4ae33453bf8b1e1baedfcd9cba23cad913ca282a57be87fed18ddb6c28e754 1756 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-server-linux-s390x.tar.gz) | 3abcaa6fda41b19a0b5e2627e93b0004759d291ea22c8698008f0924a7c8a5c2aece81c508e69e37a60980f94adb6873a68fe79a7c659641a453d706c90b26c4 1757 1758 ### Node Binaries 1759 1760 filename | sha512 hash 1761 -------- | ----------- 1762 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-node-linux-amd64.tar.gz) | 57eeae81081e06e35484b353be04f18bf5c2556175a0355d63cbe3eea80d51decfae28eb42cb5fc8907492a70e4e9bae54bd86956caea7c3a51b1fc909feaea6 1763 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-node-linux-arm.tar.gz) | 26510f1f342efa97d3a340db699f8d18d3b8430e082ff32c4596fc3efb629cdc0d427af6f749c4ed845ea2c0f2594ece7e23bc875970e5433e585812e8eef84c 1764 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-node-linux-arm64.tar.gz) | 70a73dc630df6fe60682e5379624116607a6fef571f17423de309d1ce20895eb358472809e736743740d4a2df1f708799975013d4cdc47499c5f1df3d4b2d630 1765 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-node-linux-ppc64le.tar.gz) | ebf98d17dc9ffba9ab895bf4877099563cc01ae930b9ec0342936ec53dd6b5335cb2ff01baaf620dbb9d7270e2f6831e12201c3da6c0ccbdccba30288bfb1317 1766 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-node-linux-s390x.tar.gz) | 708a9101cf73f5c78cd7d7199833b5f2f74a7cca8a4f1e0db629e3a47250a3e5338269de4b3fba703d9c084ccb4f3f30e57a88f5dedc09fe47964a0751275460 1767 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.5/kubernetes-node-windows-amd64.tar.gz) | 8aa1ad4a60edc6b677f21509eb6120dae4bd396317f28ef2d73a49986e3aafb899a5e07c7944cf5337a9b9bd514e14c3d49757c9041d5aaf80bd93f493288101 1768 1769 ### Container Images 1770 1771 All container images are available as manifest lists and support the described 1772 architectures. It is also possible to pull a specific architecture directly by 1773 adding the "-$ARCH" suffix to the container image name. 1774 1775 name | architectures 1776 ---- | ------------- 1777 [k8s.gcr.io/conformance:v1.23.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) 1778 [k8s.gcr.io/kube-apiserver:v1.23.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) 1779 [k8s.gcr.io/kube-controller-manager:v1.23.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) 1780 [k8s.gcr.io/kube-proxy:v1.23.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) 1781 [k8s.gcr.io/kube-scheduler:v1.23.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) 1782 1783 ## Changelog since v1.23.4 1784 1785 ## Changes by Kind 1786 1787 ### API Change 1788 1789 - Fixes a regression in v1beta1 PodDisruptionBudget handling of "strategic merge patch"-type API requests for the `selector` field. Prior to 1.21, these requests would merge `matchLabels` content and replace `matchExpressions` content. In 1.21, patch requests touching the `selector` field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. ([#108139](https://github.com/kubernetes/kubernetes/pull/108139), [@liggitt](https://github.com/liggitt)) [SIG Auth and Testing] 1790 1791 ### Feature 1792 1793 - Kubernetes is now built with Golang 1.17.8 ([#108559](https://github.com/kubernetes/kubernetes/pull/108559), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1794 1795 ### Bug or Regression 1796 1797 - Bump sigs.k8s.io/apiserver-network-proxy/konnectivity-client to v0.0.30, fixing goroutine leaks in kube-apiserver. ([#108438](https://github.com/kubernetes/kubernetes/pull/108438), [@andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery, Auth and Cloud Provider] 1798 - Fix kubectl config flags incorrectly setting burst and discovery limits ([#108401](https://github.com/kubernetes/kubernetes/pull/108401), [@ulucinar](https://github.com/ulucinar)) [SIG CLI] 1799 - Fix static pod restarts in cases where the container is not present. ([#108164](https://github.com/kubernetes/kubernetes/pull/108164), [@rphillips](https://github.com/rphillips)) [SIG Node] 1800 - Fixes a bug where a partial EndpointSlice update could cause node name information to be dropped from endpoints that were not updated. ([#108201](https://github.com/kubernetes/kubernetes/pull/108201), [@robscott](https://github.com/robscott)) [SIG Network] 1801 - Fixes a regression in the kubelet restarting static pods. ([#107931](https://github.com/kubernetes/kubernetes/pull/107931), [@rphillips](https://github.com/rphillips)) [SIG Node and Testing] 1802 - Fixes error handling in a kubectl method used in downstream packages. ([#107938](https://github.com/kubernetes/kubernetes/pull/107938), [@heybronson](https://github.com/heybronson)) [SIG CLI] 1803 - Increase Azure ACR credential provider timeout ([#108209](https://github.com/kubernetes/kubernetes/pull/108209), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 1804 - Kube-apiserver: removed apf_fd from server logs (added in 1.23.0) which could contain data identifying the requesting user ([#108634](https://github.com/kubernetes/kubernetes/pull/108634), [@jupblb](https://github.com/jupblb)) [SIG API Machinery and Scalability] 1805 1806 ## Dependencies 1807 1808 ### Added 1809 _Nothing has changed._ 1810 1811 ### Changed 1812 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.27 → v0.0.30 1813 1814 ### Removed 1815 _Nothing has changed._ 1816 1817 1818 1819 # v1.23.4 1820 1821 1822 ## Downloads for v1.23.4 1823 1824 1825 1826 ### Source Code 1827 1828 filename | sha512 hash 1829 -------- | ----------- 1830 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes.tar.gz) | c88f633b0b418469aa381cd39da1581236ce3e7df6f983434d4ce95fbd810a63005a78745f20f16661fdc6280d83be2cb52c8777c0ab5a8c526dea30669b463c 1831 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-src.tar.gz) | ae34f80b5a13f717179954a99bb5d0481b3b9bb1ea27e805341f6911d0cd1b3f2d58e5cd375b8b84efb0325fc13d7536f9642790aad129e8361d60f36169e2e5 1832 1833 ### Client Binaries 1834 1835 filename | sha512 hash 1836 -------- | ----------- 1837 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-darwin-amd64.tar.gz) | 6cdbaffea1ed917cddf6c4d7630eb33dddc36c7194421bb291ea7c7a8acdc235ea061cb3a8d72d84e5d314f61497594b9a5b40dbf4a69baaac88cfb48eadccdf 1838 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-darwin-arm64.tar.gz) | 200025cd65155ef8d6a854a96f8fc28f819dc3d4f7417af4c0da141d31036d48eebeb3fbfb4efd5d1adf974de54eb549f6c76f26c585b689d46484ca12902edb 1839 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-linux-386.tar.gz) | 111157e8a37ddf5d746018a4c8c8b16e86f6c2e18b228d9935b5eb2da630959e22362e3c328435ce0b5976eaba420f331c8ac0eb217d81577f0a5a956b529397 1840 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-linux-amd64.tar.gz) | 280c0b62d7b19c23b30e52e2b6d3aad676de003cbe711ee20bf03227362133782ab6c40f9bf4ac2ae264d0522a863084050e0283eb7b46d70393091994aca30c 1841 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-linux-arm.tar.gz) | 0d2c1d7091fcfa37b439e3c8ae5643385860e1cf6578fe58df85913b607f356c5e6e2b348e8f156fd293d53c8b4b520527640c5b585990397f42699c3ba0e146 1842 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-linux-arm64.tar.gz) | fd995845ebd87195b8de662097f423f6c4c71addeefd95387303be16814374eaf044894e28e84cf1906cf33569c5b486919fff97e7aa32e7528fe591c100f3c1 1843 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-linux-ppc64le.tar.gz) | f88356c7f1bf84eba54ad83de924f35f893b3f8dd6ed78518907f8f69cc85048c3b1eb176e945e8b3ecf5b2cbdd0f95985e4f5ffe91eccd3cb24050d6a3c89ed 1844 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-linux-s390x.tar.gz) | 899ae660288a6dee79e3e9b64ca8c3c37c8bcdf290748074172f546da785d805f9ed281b3f9a443a0dd1e86b8e112c8b98c0bc85592526e10ab8420dee90ca59 1845 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-windows-386.tar.gz) | 0bd0219cf5653204ac89cf246521f4ce56f89218ce4ace979504a70947a7aad9c7e783fbcf2e090996bedc90ffc3084683527ba5df76858f31727a58e41b2740 1846 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-windows-amd64.tar.gz) | aad2ae58858017484683347849d04baf3bfc7eee6984548383b3bb0150fc8cc25b1aa73bf4e1fcb9878e05cc2f71c44811f5b50b3bdf9662b79627f3ef3d3d9f 1847 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-client-windows-arm64.tar.gz) | c5d6a22f8264b38b67c316a44619095c1826e46996ffd93e4a78f6f7cf8d85f6e9835270659dbe01b0c495873e40d9373fdfafa26e5fc6d8e078407c763b22b6 1848 1849 ### Server Binaries 1850 1851 filename | sha512 hash 1852 -------- | ----------- 1853 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-server-linux-amd64.tar.gz) | cd5e90d25bd48dbbd1755eb5d328676c6424d191c2936d7477c9dde72cd374389f888bfee6ffd02836bbe136e8bf9a2cb8adcad2c650714dc6112075949c06c7 1854 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-server-linux-arm.tar.gz) | cef6eaea9e1f65cbcf8cbfd254d61e6faacd1550bb691ee5cf7f157efc61a4cce6a6abdb26c42c628706f55b062f7892ab5d40665703e22d6c1dc16dfa8e8ad0 1855 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-server-linux-arm64.tar.gz) | a7762d0b380fb06675bad6d4b987e3ddfe0c2b54ce1592c9d2c853d3a8a4d85bbfa77013e92985f3d47269010ff03fcef1166f91aa691169e920b367b3babdb0 1856 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-server-linux-ppc64le.tar.gz) | e8390ca2a4cf7d2e4b1ab5a42da4a47a0761fea200ee83626c8e81b2790cc6a20b25b090ae2b67a0a81a946a35f3468b3a09dbfabf195fe359a1a159260c424f 1857 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-server-linux-s390x.tar.gz) | 1501c640e22acbe03ec06b76ccad8bece1038039f2a7d71d9504843c8173551b04b2d144ec437d3ea1998e8ee9f25df3739e85788c738f6efca274dac919a947 1858 1859 ### Node Binaries 1860 1861 filename | sha512 hash 1862 -------- | ----------- 1863 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-node-linux-amd64.tar.gz) | 9f75ec2e71469bf5d53f0ac305128d7b685f7d2daf4dd3218a8c89b36bd3f3e73f69696ece76bb4ed1254e1fcd89abc1fe49896db73222a7ce8c0795afd37e93 1864 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-node-linux-arm.tar.gz) | f9085bf9b750dbebcf713aa4b5166e65aa8a313be468e1e8014e79a516018b930a9dd4e093fbb632a1538e59c6b42017c0c15050fe67407c92e7141ae073af49 1865 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-node-linux-arm64.tar.gz) | 5e6f30f07f1f49092c2201303d5f3843343c1453cbcb1603617df9aa43bf4549581afd9332d634127320fa543747f730c57c00f437cde3f912b6476b571e5bb8 1866 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-node-linux-ppc64le.tar.gz) | a63da0682d139a9ccc261fef8e9b944b31d8cf020416f5fe3216ea5da6e2b76c65778064df51009e8a4630f41c811e9c082682a5731766647b372de2b04f1035 1867 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-node-linux-s390x.tar.gz) | 3c7e7295f1b133a0469f9a70a56891cef9d990e7959c75f4c5dfa1dbe8f2e6bc689743b32cf5d5ba1bf854c5dac812328f4fcff6606a97c77ae06c24848de2e7 1868 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.4/kubernetes-node-windows-amd64.tar.gz) | 434c0a397a10c06bbed9228944741605a97eee3c0fad6f4de341a5ac882439879433f84a5e269019ad599d440218a1277bac0dbc38e01ff294ec7044a1283076 1869 1870 ### Container Images 1871 1872 All container images are available as manifest lists and support the described 1873 architectures. It is also possible to pull a specific architecture directly by 1874 adding the "-$ARCH" suffix to the container image name. 1875 1876 name | architectures 1877 ---- | ------------- 1878 k8s.gcr.io/conformance:v1.23.4 | amd64, arm, arm64, ppc64le, s390x 1879 k8s.gcr.io/kube-apiserver:v1.23.4 | amd64, arm, arm64, ppc64le, s390x 1880 k8s.gcr.io/kube-controller-manager:v1.23.4 | amd64, arm, arm64, ppc64le, s390x 1881 k8s.gcr.io/kube-proxy:v1.23.4 | amd64, arm, arm64, ppc64le, s390x 1882 k8s.gcr.io/kube-scheduler:v1.23.4 | amd64, arm, arm64, ppc64le, s390x 1883 1884 ## Changelog since v1.23.3 1885 1886 ## Changes by Kind 1887 1888 ### API Change 1889 1890 - Fix OpenAPI serialization of the x-kubernetes-validations field ([#108030](https://github.com/kubernetes/kubernetes/pull/108030), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 1891 1892 ### Feature 1893 1894 - Kubernetes is now built with Golang 1.17.7 ([#108100](https://github.com/kubernetes/kubernetes/pull/108100), [@xmudrii](https://github.com/xmudrii)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1895 1896 ### Bug or Regression 1897 1898 - Fix Azurefile volumeid collision issue in csi migration ([#107575](https://github.com/kubernetes/kubernetes/pull/107575), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 1899 - Fix e2e test "Services should respect internalTrafficPolicy=Local Pod and Node, to Pod (hostNetwork: true)" ([#107902](https://github.com/kubernetes/kubernetes/pull/107902), [@xueqzhan](https://github.com/xueqzhan)) [SIG Network and Testing] 1900 - Fixes a regression in 1.23 where update requests to previously persisted `Service` objects that have not been modified since 1.19 can be rejected with an incorrect `spec.clusterIPs: Required value` error ([#107875](https://github.com/kubernetes/kubernetes/pull/107875), [@liggitt](https://github.com/liggitt)) [SIG Network and Testing] 1901 - Fixes static pod add and removes restarts in certain cases. ([#107761](https://github.com/kubernetes/kubernetes/pull/107761), [@rphillips](https://github.com/rphillips)) [SIG Node] 1902 1903 ## Dependencies 1904 1905 ### Added 1906 _Nothing has changed._ 1907 1908 ### Changed 1909 _Nothing has changed._ 1910 1911 ### Removed 1912 _Nothing has changed._ 1913 1914 1915 1916 # v1.23.3 1917 1918 1919 ## Downloads for v1.23.3 1920 1921 1922 1923 ### Source Code 1924 1925 filename | sha512 hash 1926 -------- | ----------- 1927 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes.tar.gz) | 339d208b86206272494d4f31a384fd8430911a1f8205d4a73605f412b4653fd816e79653bd0a0dacf52c9b9f6a3194279cc1059683c2ffd560c1ad3fa185e20f 1928 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-src.tar.gz) | 9530d46878aff36b26b6e8f8bb04c53eb402bd822851f5aa65a2ac6e46064f67820a63a2153ffd317a0b6fa3383cdbc58fe55484a0dd3197862f3133fede5a5b 1929 1930 ### Client Binaries 1931 1932 filename | sha512 hash 1933 -------- | ----------- 1934 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-darwin-amd64.tar.gz) | 96810fc0f294bfe412f06125b257ae4e7d4ddc7d11247435a3f500830af334560b61399ba8ac9753c63857b42a7fd13c8f7d55a927cb7981aecd886a84ae8a90 1935 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-darwin-arm64.tar.gz) | c9d1bad58e46908d190760fb1615188180ca1c734ab4137437e62baa76d92e3b79acf8c39e693c5b947bf650bf894f5fb96651d4b617d35992cd82c1d1f64709 1936 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-linux-386.tar.gz) | cbc63934798cb57f0be188346e852fb9ca8da071c38c5b75c8199b5637bc8df7df9019855607c72ac634d7944e86db0b9485b29b36365d93f8b85f2703b0f3c9 1937 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-linux-amd64.tar.gz) | 7ee6292a77d7042ed3589f998231985e82abd90143496a65e29b8141dd39dced5f9cd87a7eeba1efa4dbf61e5ddec9e7929c14b7afcdf01d83af322ddf839efb 1938 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-linux-arm.tar.gz) | 36147a76eb16869bc07608f947b78ff15c3d60b73deef208e5c135d93e1a48d17e5f8a447894a450a6cf1a1d5f058b07463dcbf1d0120d17133d2d98cbce2444 1939 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-linux-arm64.tar.gz) | fb66a9735f40e2df40388df1f8e17aedd1ac87f7190d76e3eb2a5dd1a11494be56ae312be5aea0f7613b826f3b9b3ee923eef9736b689415605de351eb8861df 1940 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-linux-ppc64le.tar.gz) | 6b3103ade6e0d7d918461ae33978636343b3d122dda5a68287c21eaba7c6abc2de49277828640e4d25904134aa4311665615ac78783b5fec48314bb3ce09a3ee 1941 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-linux-s390x.tar.gz) | 13edeefe00b9d9c151ba27a4190a0dfe5fbb7fdf409a83787eb90a0f38be1cfffa7be50b180d5fe96acbdb6ff335ef9ba8c90fb828f4e9953e4798aa3b20963c 1942 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-windows-386.tar.gz) | 4c761bf7ddf59a980cc800602c5ae1379d9b39b3a15fc35a0a9b2b34fe18150dfea5e6c84b2548477df4a563ffd3290fe84cac80e5f9bc6f07863a2efdfe049e 1943 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-windows-amd64.tar.gz) | 92cc39b07c62ce5c436f167bb8929f678e362484d7c40b7ef76562ea61db93a38e811325e0992b42349edfc5136b110aff169a6423a29408e65266cbd204c8f8 1944 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-client-windows-arm64.tar.gz) | 71a136733fc032af8c825c402355b73e1049f3706c8b88fc3c7c78da0d3e0e6e7eb1fea455a833092ec7d9e12e27230c1097fad1129f487fcdcc4e54b81cceb2 1945 1946 ### Server Binaries 1947 1948 filename | sha512 hash 1949 -------- | ----------- 1950 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-server-linux-amd64.tar.gz) | 667bc04778070685e5fb5b6281fe78263c5081af0613adfe9a68df0695210cb2273e89a1d37a27e4cbf947b9e565ef7697d8b90ddfba23aeeb4c9f8474a373c5 1951 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-server-linux-arm.tar.gz) | dd3dbb478185819a4783ac5ed923282ce5be6d9595228226dd8c8c447ac78e5ab7b4558c452c39767df571dd0669552ce096a6bc4d0ae993511e5c3274759961 1952 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-server-linux-arm64.tar.gz) | 1eab0b0102cf6635a3e92249d95c08ac36baaccc14a612f512009e0aff28d563b4fd818deb4aaa471aa8399a3a5bb67b10b484dcf0d3e54615d906a1bd861cdd 1953 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-server-linux-ppc64le.tar.gz) | e648010752a1db8d23119ffd44b672f67040e4f841b014df699bfa328c32cf97bb928a1c17d1719098a229d83844a593a9cfb1d0833f562a7d64e71ff20120d9 1954 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-server-linux-s390x.tar.gz) | 0da7c96e2360f8272dd6cab9da7a3a6b516760b39a9fcd38e86365c2b6f7bff0e839892b58910921aa25d8dce0c94b46cee58bd686b9369995641b6771fa0191 1955 1956 ### Node Binaries 1957 1958 filename | sha512 hash 1959 -------- | ----------- 1960 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-node-linux-amd64.tar.gz) | 9fd17ed04dc8e13ba5b4d67ec657b8afba721c344bd9785669af3def481dcbd8a2eecb02e54e5eebd0559645c6e819f757c49de731e53073f06a12d871e569eb 1961 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-node-linux-arm.tar.gz) | 8c934bc5b3a545a8a5c2fa9f6df7358127125509ebd9ddf1b074121f78c6520ff0e4af9217b9930dd21b28c4960946673914f35060283b83606073d308ef28e7 1962 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-node-linux-arm64.tar.gz) | 6319071775767b4eab400f3068ec4c0901756ccb79db63fa9ed6754047bacbaec55cbef366a92e057ca610eb6f20d6e4341b9a1f7a36ced44435d07bdb0af0f3 1963 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-node-linux-ppc64le.tar.gz) | 8bf110dac7e4e61ca9a2a513a6e296bf36bfd8dee85e7c2c46f831e4eacdeaf6b238b361b044b5a7de0f14f1d735f41e8e169cdb676ae4f4c109da457593918f 1964 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-node-linux-s390x.tar.gz) | 35afc0d3f31b6795a280cb7005cc7c5253e897758aba36f4d4558a0ffc2b34ac4f0e7a1e7a42ffc3c67ca1a52365bd3052ea554e43b8cca4bcf2ab38c6ea7929 1965 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.3/kubernetes-node-windows-amd64.tar.gz) | 8d687018bf4b70065d4871406702d57f0ef14abb6c8e8bd7635d2d94f8a56aead9a641ede4477e34534bc705e76bb94cec10dbb9414c5885ad0a5d07d1105401 1966 1967 ### Container Images 1968 1969 All container images are available as manifest lists and support the described 1970 architectures. It is also possible to pull a specific architecture directly by 1971 adding the "-$ARCH" suffix to the container image name. 1972 1973 name | architectures 1974 ---- | ------------- 1975 k8s.gcr.io/conformance:v1.23.3 | amd64, arm, arm64, ppc64le, s390x 1976 k8s.gcr.io/kube-apiserver:v1.23.3 | amd64, arm, arm64, ppc64le, s390x 1977 k8s.gcr.io/kube-controller-manager:v1.23.3 | amd64, arm, arm64, ppc64le, s390x 1978 k8s.gcr.io/kube-proxy:v1.23.3 | amd64, arm, arm64, ppc64le, s390x 1979 k8s.gcr.io/kube-scheduler:v1.23.3 | amd64, arm, arm64, ppc64le, s390x 1980 1981 ## Changelog since v1.23.2 1982 1983 ## Changes by Kind 1984 1985 ### Feature 1986 1987 - Kubernetes is now built with Golang 1.17.6 ([#107613](https://github.com/kubernetes/kubernetes/pull/107613), [@palnabarun](https://github.com/palnabarun)) [SIG Cloud Provider, Instrumentation, Release and Testing] 1988 1989 ### Bug or Regression 1990 1991 - Fix: delete non existing Azure disk issue ([#107406](https://github.com/kubernetes/kubernetes/pull/107406), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 1992 - Fixes a regression in 1.23 that incorrectly pruned data from array items of a custom resource that set `x-kubernetes-preserve-unknown-fields: true` ([#107689](https://github.com/kubernetes/kubernetes/pull/107689), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 1993 1994 ## Dependencies 1995 1996 ### Added 1997 _Nothing has changed._ 1998 1999 ### Changed 2000 - k8s.io/utils: cb0fa31 → 6203023 2001 2002 ### Removed 2003 _Nothing has changed._ 2004 2005 2006 2007 # v1.23.2 2008 2009 2010 ## Downloads for v1.23.2 2011 2012 ### Source Code 2013 2014 filename | sha512 hash 2015 -------- | ----------- 2016 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes.tar.gz) | f30d444bd0fc62bd8f7d352dacbdc2fe8904707f3c4f6d719e62f6c9509d5d544a1b26964228c3ff29b9c451534d9f85fe25a60b09b332fa5291e542720cfa05 2017 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-src.tar.gz) | 6a54e73972672415c9d1472764f6f266700da807a6ee9cd530e28a5158d33280702f3d94948e914347e32df24d9bebdc0d4627adef5221fc7bd3b12e2f8d2a93 2018 2019 ### Client Binaries 2020 2021 filename | sha512 hash 2022 -------- | ----------- 2023 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-darwin-amd64.tar.gz) | 7371cbf87768e49cbc7ef2776fe037eae2809eefefc5242733da119328d49facf714cb04113c15cfbda99b6b5f62fb82b66951c04011fcb642bd056adb11acb3 2024 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-darwin-arm64.tar.gz) | a5c8aa760e1cd94e469beb1e73c5abc786bc294278b2e92082dc1afc7ce6e3f8c7f157752331c45579738642d3c159c23d0ac303e0838157cf6fa06d649ef800 2025 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-linux-386.tar.gz) | 84af3cd829626a8737e38650dd231e158edce11706612f357e19c2e8ef316b31239d6ab82c5ba565c1bbf2975556eeefb3ca6c757d17d27303ff6cf1ad4b9f0a 2026 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-linux-amd64.tar.gz) | c8653aa2bce09a29041b7347ce2d45710abd8bc3cfe79265e0aa04a24c2028344f0b280f52b2858e869bf997cd0e71b6ac8f22ec8a2b4b39328e73339746f565 2027 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-linux-arm.tar.gz) | 35a00f6296ab70d0af2838915b848b9f0df5b778935ab8089b4b180c7406b958a1c909adc1bbdf12857d0c75b18be3c236c617f74d392d3eb2f8cb85eb862eca 2028 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-linux-arm64.tar.gz) | b00539ec1d993e272b77d4ed3a46be743af645cecc6320c9a017c0c5f4f48dd0272377f67e993c2f06aae0fdeb6a174da901be9ff1e9f6c8cd311e16c5cf60ce 2029 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-linux-ppc64le.tar.gz) | 135e2aa8ae000ac6fe88ec8afa0f671147b9d8936def510a53d2a456191805daceaef8152d22d91a57bcf4bc7e8b47e587440ed260d3f8b2d5013c90d125fcbe 2030 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-linux-s390x.tar.gz) | 860ad0d3eb064e1ca3b2ce74a296fde1fffe3e620ddfd579f7d022032419bb8f0c7300565bd94d6681cc50ba2268bb2d64a94c1cc3b9b999f0c031de1b92999b 2031 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-windows-386.tar.gz) | 907c3043a1f06912238ee5d91f7d76d9bbc5417363deb3d9f2cef86bd79e72ad4f7d9cbeaaa7003afa5570bf2a689a8d619ef60e5cb89e61d53eb5ef373102e7 2032 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-windows-amd64.tar.gz) | 1abd5aeeaaff5884238ee39022ef18b91518026c93e4305f25de9d2cc2136fbacb9799deaf138c691ad82928ac22f7156046793b04b7240625b2e5043b65e9ee 2033 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-client-windows-arm64.tar.gz) | ab5617a9a6b154af6a2329523c2fd356583b8266cf9dd512dccdab5a21a9f82ab5eab4802d65e0bc191cc492215e2d5fb850f6f8c02c403635cd0efbe8349450 2034 2035 ### Server Binaries 2036 2037 filename | sha512 hash 2038 -------- | ----------- 2039 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-server-linux-amd64.tar.gz) | 7f4cc97250be176a9af0136c25a549fc491caa0ef6300ccca798c599c5ed5182c08ebba1d8f669ddf3011633cd7b12523d7e37abc65faa369defdde9e351eb22 2040 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-server-linux-arm.tar.gz) | 572fbad7f0edcb1e1294d81e222f4add78d302d80ea28d7456b8020b53f01fbe54bee424d4807ed05d6eadaaab46a323faf36b3a3da7427bdc075749c3365d9e 2041 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-server-linux-arm64.tar.gz) | 971ce29019cf248c167c27fd081458fde613f7f92c5fe4ad3816eb12ab157c30eb78105b999144d0b31e7093c0f826b1df9cc275b120d0ac269062726669d0c8 2042 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-server-linux-ppc64le.tar.gz) | e1cc8146a2c2a0b5774a3516548323aed948dd03e93545ec11c23be8bc5b23e3395b593a656dafb36711e7b368b3cf1c8dd2b55bf5a486fc3ac4875bd011dd22 2043 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-server-linux-s390x.tar.gz) | c3f766166e64f878c9077c1d070326da0ce71881b40204d60596921d62d568282f6527137e0f362ff3895ad156ab92dc026ed19f305120872aa385386033e6cd 2044 2045 ### Node Binaries 2046 2047 filename | sha512 hash 2048 -------- | ----------- 2049 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-node-linux-amd64.tar.gz) | 2fab5c395fbf5f88e56430c51979670db0be5119d79a8a74846900601a313917b91615efabe49a92b3b8ad2f11d2dc5892f701549b0debeb16012c3705612d01 2050 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-node-linux-arm.tar.gz) | cfe8e40981aec67fbe9cfcba4af18b1ea4f5b4f34e5b9b11cdb6dee076ad8be2b351d0871b1e9d715cec4ca629c87a5cb598a70b6f8e1dcd15486c1f387a5a42 2051 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-node-linux-arm64.tar.gz) | b75d8d1e0b9ba1b06b3f08d67ffbff01784eee8973797d0f1565efda4d61bdb89fbea45c385ca3224fc75237ffe6a41920fd1f51b73d5021007ec9e4ef88af73 2052 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-node-linux-ppc64le.tar.gz) | fc47a39b6cbe9d4237740d060234c065c7bcf33fbb10b3cffc670b6f7eacfb9f44c2695c620a42fd37a2e30e375f29d206ef4d4fc8a1ded7f57bb9894c85b178 2053 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-node-linux-s390x.tar.gz) | 1e9b0d5197b436a14a1e772027c951ba580c5a047acf201102c585190eefc8a8871fee8e20a40efcc01c0475776540ec8cb8f09aef43c149e335a722670cd855 2054 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.2/kubernetes-node-windows-amd64.tar.gz) | 37d6edc06bb5a555c0594875f917a80f42486e59252c0a8b813b3a935352306a19178c4b5c0af4251b40be58fbe3b99377387d1c861e32f4599ef0e275bd4299 2055 2056 ## Changelog since v1.23.1 2057 2058 ## Changes by Kind 2059 2060 ### Feature 2061 2062 - Kube-apiserver: when merging lists, Server Side Apply now prefers the order of the submitted request instead of the existing persisted object ([#107567](https://github.com/kubernetes/kubernetes/pull/107567), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] 2063 2064 ### Bug or Regression 2065 2066 - An inefficient lock in EndpointSlice controller metrics cache has been reworked. Network programming latency may be significantly reduced in certain scenarios, especially in clusters with a large number of Services. ([#107167](https://github.com/kubernetes/kubernetes/pull/107167), [@robscott](https://github.com/robscott)) [SIG Apps and Network] 2067 - Client-go: fix that paged list calls with ResourceVersionMatch set would fail once paging kicked in. ([#107334](https://github.com/kubernetes/kubernetes/pull/107334), [@fasaxc](https://github.com/fasaxc)) [SIG API Machinery] 2068 - Fix a panic when using invalid output format in kubectl create secret command ([#107347](https://github.com/kubernetes/kubernetes/pull/107347), [@rikatz](https://github.com/rikatz)) [SIG CLI] 2069 - Fix: azuredisk parameter lowercase translation issue ([#107429](https://github.com/kubernetes/kubernetes/pull/107429), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage] 2070 - Fixed a bug that a pod's .status.nominatedNodeName is not cleared properly, and thus over-occupied system resources. ([#107109](https://github.com/kubernetes/kubernetes/pull/107109), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling and Testing] 2071 - Fixes a rare race condition handling requests that timeout ([#107458](https://github.com/kubernetes/kubernetes/pull/107458), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 2072 - Mount-utils: Detect potential stale file handle ([#106988](https://github.com/kubernetes/kubernetes/pull/106988), [@andyzhangx](https://github.com/andyzhangx)) [SIG Storage] 2073 - The feature gate was mentioned as `csiMigrationRBD` where it should have been `CSIMigrationRBD` to be in parity with other migration plugins. This release correct the same and keep it as `CSIMigrationRBD`. 2074 2075 users who have configured this feature gate as `csiMigrationRBD` has to reconfigure the same to `CSIMigrationRBD` from this release. ([#107554](https://github.com/kubernetes/kubernetes/pull/107554), [@humblec](https://github.com/humblec)) [SIG Storage] 2076 2077 ### Other (Cleanup or Flake) 2078 2079 - Updates konnectivity-network-proxy to v0.0.27. This includes a memory leak fix for the network proxy ([#107037](https://github.com/kubernetes/kubernetes/pull/107037), [@jdnurme](https://github.com/jdnurme)) [SIG API Machinery, Auth and Cloud Provider] 2080 2081 ## Dependencies 2082 2083 ### Added 2084 _Nothing has changed._ 2085 2086 ### Changed 2087 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.25 → v0.0.27 2088 - sigs.k8s.io/structured-merge-diff/v4: v4.1.2 → v4.2.1 2089 2090 ### Removed 2091 _Nothing has changed._ 2092 2093 2094 2095 # v1.23.1 2096 2097 2098 ## Downloads for v1.23.1 2099 2100 ### Source Code 2101 2102 filename | sha512 hash 2103 -------- | ----------- 2104 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes.tar.gz) | d7b53be1a9695143b780fb9ff1271c65dd1584e09ef77fe5aa3db4f965a9a7a8b59af8981b3dfeba1f89dd48a81e30f1cd4d443b7b9bee9f1695b3346b41c8fc 2105 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-src.tar.gz) | 00e07c8f2b42bda04f780f74fb5625f52d7a16b99424a0f7a4c67101923eabe495f440f66317ef151b9ae48253c0dd2fa4730b8a4e28b86f13c7741e96cdaaa8 2106 2107 ### Client Binaries 2108 2109 filename | sha512 hash 2110 -------- | ----------- 2111 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-darwin-amd64.tar.gz) | b7e858cdb049e710d3961a791d0ae4b0c2309b024f98a80dd470e4e6a2ab30bd2aae72eed57561af27dafdb317617d60c8b7f454c2a0873bd2801751d8cc0260 2112 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-darwin-arm64.tar.gz) | df441b4757c7ccc3e13145ad264c1de8101e442cf267b3537d2506e9c62f5da4738fca14967367c58933554fb5deaa47cfed146bfb03e0b81b0fac656696b4cd 2113 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-linux-386.tar.gz) | 3dcc223be7562a78d01e491fe101196d9c644490e4d9a11f5a2314cd25c0f1870ed1dc0c2b872c24d1802348a3d209d0e1304ed16a5cac76d121090e914f4191 2114 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-linux-amd64.tar.gz) | aaf6c4f2f65b27902ce02069aa5a7c5b195099deb522fbe7638c5458fc1ba6c2fbe2eb00fa18edc952989dfc27c7a252b37792987c55dc044b88d4e350569891 2115 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-linux-arm.tar.gz) | 5091b2731725a53a9d0db7e45dcd3f534978f6f0361dff13f36e8c22259506df622e25d0c3b81867328a3314d98d9545b351bbf49ff45f9fc5444f2f67359546 2116 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-linux-arm64.tar.gz) | dbd3ce1ec9cc3e89a0510ce3809966b38fbd95e538b9b9426b9c303e1dbb71eb44c1446bf0b4c70a58d9f1c29a75bff71543fe0e8c724a5d5a03082eabdd9f02 2117 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-linux-ppc64le.tar.gz) | 677e733f714b148478b9576bd8fe56e78e7517b2fcc5b1d276e92c958933002b9565bc8bdc8f22c2c41857c9b1e92a008baf48477d31d485cfca6a68a82f32ee 2118 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-linux-s390x.tar.gz) | 8019fae5c10b146594e300fb5591eba45df7a637db4bcbea8943bbef37d06f2d349f338c4b48500ad92e477a869c29e980dfc80e42d2a759ad62236757c53718 2119 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-windows-386.tar.gz) | b9d3831e78220abc15b0384ab3739f4ba105e324270012a2519ffae4b22c963f2eb602963b68f6353807ab42ae1a1699ea82114edb5e2650af59bd7b6c700288 2120 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-windows-amd64.tar.gz) | d661b48b7d5dedadcd644a9dd4eb99b756784e43026a39cd837194f8eae5d4356fe0c4de7f4b85ef7ef6fb27a5bac8b9cc99ab92e8bb5bc092f38d3d781e9921 2121 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-client-windows-arm64.tar.gz) | 833ba3c0afafc1fa5fd40d20e0450eeb591fe3eb2b73cbcea74fcc029cdf17caabc3a2ae27d6f3227b30c90c31c0d9a2c57656ed3f8fd1173d38938b98e74374 2122 2123 ### Server Binaries 2124 2125 filename | sha512 hash 2126 -------- | ----------- 2127 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-server-linux-amd64.tar.gz) | be1b895d3b6e7d36d6e8b9415cb9b52bb47d34a3cb289029a8bce8a26933c6cc5c0fe2e9760c3e02b20942aa8b0633b543715ccf629119946c3f2970245f6446 2128 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-server-linux-arm.tar.gz) | e5518962fc1543cfba096693c9ca3fb026a11395dd6eabdf50fc577c598e6c546860a96def681681dbc84025e8f06e3bbb13b28a1ba829bee359b779e884f626 2129 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-server-linux-arm64.tar.gz) | c546831cca738c3178ff464891d15f84c10d754c1c9b70742b1fa638d108afabf320aabae9dcfd1ec2e6e77439e5151c561ca6a2cb8989cb533035f48509bda8 2130 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-server-linux-ppc64le.tar.gz) | 6e72592a8ab51d6e7875c327159918a737deca88e168574b0dae77c08e0325acc575b62bf3c166e3c6da438fdfc2f996911e134de0c57a1c5001ce29d3b7d97e 2131 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-server-linux-s390x.tar.gz) | 1092c009c518f28b089c962e33c73f97af2226e5d100856a3ac996f47da4519a4c450dd98ac5d78519fe12e47b955df98675bf4ecbf0d8a722d994d5777b4107 2132 2133 ### Node Binaries 2134 2135 filename | sha512 hash 2136 -------- | ----------- 2137 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-node-linux-amd64.tar.gz) | 77bdd46d93d6dcdb2e5a66ecf0e5f9b6fffe49c53920a36c2e0c4ceb3625631a4a09662dbdf4c170bc5ee35968da257b80048d21d4d1f7302123a549fe99429d 2138 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-node-linux-arm.tar.gz) | fb619f27a72cb014e5e97c287d768fbd23bcd21dd8f5d2cc1a4e4d7a21727781ba914a1af9fc33436795e7e7c835bca3e9afa4cbc5d691691f39fcb5108617c7 2139 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-node-linux-arm64.tar.gz) | 6f3a98d8929f1e7088ad8751dea6a9fe588ff6a7c40d29b5723ebc47f1badc2130ff4a72c6b2fd3bc0c9dce2b93329ee3c7cef3be93fdfcaf82162bbe0314adc 2140 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-node-linux-ppc64le.tar.gz) | 3c8b14d8680612af73f56dbf8b1de4c7c45addf98cda1aa7cdf0b8a0e8f12a95dffe336fd3aa4cd93e072cda5dcde1a88da3034ce8cf5bd11718800596342986 2141 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-node-linux-s390x.tar.gz) | 6f8a30f6ea0114156e636208ab09de9763ed08844d1278f7d034575ce0b4b486aa3fd3cec8992651d6de8263b66aa8285a1180ff2f6011a2904eabcd872eaebb 2142 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.1/kubernetes-node-windows-amd64.tar.gz) | a0cf768d92b51d370842dd7b819c20d6fac2bee955763ead4700f095ac0bead030a9a4b2347987e5ee862402f3d5a7a306343229777011be21544286a2ad9448 2143 2144 ## Changelog since v1.23.0 2145 2146 ## Changes by Kind 2147 2148 ### Feature 2149 2150 - Kubernetes is now built with Golang 1.17.5 2151 - golang.org/x/net to v0.0.0-20211209124913-491a49abca63 ([#106835](https://github.com/kubernetes/kubernetes/pull/106835), [@cpanato](https://github.com/cpanato)) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Release, Storage and Testing] 2152 2153 ### Bug or Regression 2154 2155 - Kubeadm: allow the "certs check-expiration" command to not require the existence of the cluster CA key (ca.key file) when checking the expiration of managed certificates in kubeconfig files. ([#106931](https://github.com/kubernetes/kubernetes/pull/106931), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle] 2156 - Kubeadm: during execution of the "check expiration" command, treat the etcd CA as external if there is a missing etcd CA key file (etcd/ca.key) and perform the proper validation on certificates signed by the etcd CA. Additionally, make sure that the CA for all entries in the output table is included - for both certificates on disk and in kubeconfig files. ([#106926](https://github.com/kubernetes/kubernetes/pull/106926), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle] 2157 - Kubectl: restores `--dry-run`, `--dry-run=true`, and `--dry-run=false` for compatibility with pre-1.23 invocations. ([#107021](https://github.com/kubernetes/kubernetes/pull/107021), [@liggitt](https://github.com/liggitt)) [SIG CLI and Testing] 2158 - Reverts graceful node shutdown to match 1.21 behavior of setting pods that have not yet successfully completed to "Failed" phase if the GracefulNodeShutdown feature is enabled in kubelet. The GracefulNodeShutdown feature is beta and must be explicitly configured via kubelet config to be enabled in 1.21+. This changes 1.22 and 1.23 behavior on node shutdown to match 1.21. If you do not want pods to be marked terminated on node shutdown in 1.22 and 1.23, disable the GracefulNodeShutdown feature. ([#106900](https://github.com/kubernetes/kubernetes/pull/106900), [@bobbypage](https://github.com/bobbypage)) [SIG Node and Testing] 2159 2160 ## Dependencies 2161 2162 ### Added 2163 _Nothing has changed._ 2164 2165 ### Changed 2166 - golang.org/x/net: e898025 → 491a49a 2167 2168 ### Removed 2169 _Nothing has changed._ 2170 2171 2172 2173 # v1.23.0 2174 2175 [Documentation](https://docs.k8s.io) 2176 2177 ## Downloads for v1.23.0 2178 2179 ### Source Code 2180 2181 filename | sha512 hash 2182 -------- | ----------- 2183 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes.tar.gz) | `850f92f4a4f397773ceabdacdb0513fa3cd2eb8867f7e3697f42bc595c3c710f81a8b9b34679d783ca2e1900dd272e0af209126cf55719e321af8da04a4b1c2b` 2184 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-src.tar.gz) | `ee53eb3b32bc4745a3f58dd0af1a8f4157e74b71b896eb39ae0658f6f3d0497b8a1334205cc19a3fe1cc7056b7606b0c745e89860f73b14ffaf4ed7bf9b6ef11` 2185 2186 ### Client Binaries 2187 2188 filename | sha512 hash 2189 -------- | ----------- 2190 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-darwin-amd64.tar.gz) | `ec7acc668cf32ec2ecf9ff515096cfe0d421c096522f4d4f6dd5504046051d6b4a15a130aab67e0d545078b26c1a0d27f2f567b1f4ac68b76324e15a216799f5` 2191 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-darwin-arm64.tar.gz) | `5c8dcc6c847d44bef1d739015627369b8b7f0b27d96bb0264bac1ea029d3e247565639cdfb9041e28d74f27c7650661ec275e452ede448181c454d77916fc432` 2192 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-linux-386.tar.gz) | `a94aa935b348dfeee09b47f3a34e8cb7b2d7213dc28e8df189a4b8438a317cfd575d97ae09651fb6a46a528cd00f0808c6e6c95d9e176837b5be463620593acd` 2193 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-linux-amd64.tar.gz) | `82574d81696693510d8becd2a2319d517dda2b424b63c5525299ed24c9f4abe1b1803fc799b2a75c650175a5a70ef03eab6068e5bf2a286a43fcaabee9681747` 2194 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-linux-arm.tar.gz) | `8dd15ed487883f76ed869458df3b5e859518491ac5b3aedb7ec95fd6c8ba1bde081859e0a7c171d9674a773b655db67975dbab9b842d1c0864a300341e58035a` 2195 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-linux-arm64.tar.gz) | `125f51a712fa4cad241e84a57baa4bc7950b4977bb4f7275ec21e82758ea90137d00d39841061cd9c4665144a2cf8ab87b99e185a64152f9682ef524266c45a3` 2196 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-linux-ppc64le.tar.gz) | `ebf7130485c33a59fc81c2a6b8d19b847470f57f4be49ead06e0405a9b34891de456199a87ef105b0428f3d9767ac39f83b8199171ba60ab7b7c538b0558d1de` 2197 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-linux-s390x.tar.gz) | `dc0a122755d096f18de5d33a7596bae3c8cce058d22999c6754377cd074b8d69f597c62b5468cc80eda4144edea026b9b946b522b144e8d9ad66874de2d0bd9b` 2198 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-windows-386.tar.gz) | `d64f72835dca883e666fc44f80db6b75467b5f952bf40f241b8b1034a8b7565aa101b3d8d9be4b47523cf22a1bb51db3924376405fdf40cd3cab688c5e9db21c` 2199 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-windows-amd64.tar.gz) | `915b7e23517dba67db9aa8b20f18f3451897fe7ab2bae1cd64bc22810e38efa3f2f62483ba377c6d1fed738de24874c2c14ef772f02f201f41a35d80eb2f872b` 2200 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-client-windows-arm64.tar.gz) | `2702663c0bc4316e83573c6c262040e72c13c6ea50b9dd042dd8d375e719e8729db759db3bffa8bd6a838661278e02a3047ce402b0348ae082393b37643047cb` 2201 2202 ### Server Binaries 2203 2204 filename | sha512 hash 2205 -------- | ----------- 2206 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-server-linux-amd64.tar.gz) | `633cba8102648735b93d91a9840a39597b3242d2489081e71b0131a9bf246e2f52060bc8f8abcc2154f39c1e2410dec5e8b189d220743b55f861062c86edd2f8` 2207 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-server-linux-arm.tar.gz) | `39eb3f85a46f6c71ba70ffa391e706b6c57c8f9fb7eea95960f944a3fee7883da56e3dca9eec7f3121800911e45681b2bcc78c4585ce7081720a8658e1837f47` 2208 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-server-linux-arm64.tar.gz) | `91236a70b0ff67b54c939215ac71a7e03e4202e71d8b11f687fc6406eb54da6271e8f095a4e812b3aaad23275e0a04b4cfa7bfa8f455a98ccf8c5a22b8b5e59f` 2209 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-server-linux-ppc64le.tar.gz) | `a41f590fcc271861d73cae14032c51d7674efba48f160550da8be7095240be39a59bdd8012886eb94afa8062576e43643a76c2796ce847d6df33d7147a807e9f` 2210 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-server-linux-s390x.tar.gz) | `2bb2d3087e911e5c296ae194697190470606c1ac761fb3e69533492109d012e756428daac93e89665f6bbac7c4b4fd1ab9e554718244234f43c6e7219c396eff` 2211 2212 ### Node Binaries 2213 2214 filename | sha512 hash 2215 -------- | ----------- 2216 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-node-linux-amd64.tar.gz) | `4b0d9188914f3b8dbb7cab384b9f2a63b5d0f53b78c7c9d4920613e780d17afffd95809eb828753556048481634b5a5bd0b89b50360e4d3f2a37d21bb88d5a36` 2217 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-node-linux-arm.tar.gz) | `039c77a9a5f7e12d826ded7f8590b9f63c79863fcc463e150848ab6304de171aebbdc948f9398478b96a763f982d02f955f8327f651a04e8caced4c6fd2b8e12` 2218 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-node-linux-arm64.tar.gz) | `d60c740f2f5b2ffe95bc9d1ea0e26ca6af1aef718d5c498339eabe6a9cc1cdff8cb69f9cc0f2389adb1eaa3dacdbcaacae3e19fd27ce035f561813d8f29b5f90` 2219 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-node-linux-ppc64le.tar.gz) | `163b33fcab0226e950c2d2415ece74ac840c592ede2e7276babab466ef3ef1cdbee95102ca4cbefeb836ce1b2897d19b0534967dbd0dcb6706753b453adb27eb` 2220 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-node-linux-s390x.tar.gz) | `c01db605cc3b744a6a13962e318f94ae66eb221d4ed3758a868452bc610cdcf79297332e158cf9e88002e3ac72c9489b7c27b001ba40df1852c6ba098afb9586` 2221 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0/kubernetes-node-windows-amd64.tar.gz) | `d5431cd60990bd56649ce11e3c5a72b92a733e6abafaee6517cec333a3dcabd6469ebd4e3d9b052d250ecb35a0c1dcac16cd841612d73886438731f0f817e2d1` 2222 2223 ## Changelog since v1.22.0 2224 2225 ## What's New (Major Themes) 2226 2227 ### Deprecation of FlexVolume 2228 2229 FlexVolume is deprecated. Out-of-tree CSI driver is the recommended way to write volume drivers in Kubernetes. 2230 See [this doc](https://github.com/kubernetes/community/blob/master/sig-storage/volume-plugin-faq.md#kubernetes-volume-plugin-faq-for-storage-vendors) for more information. 2231 Maintainers of FlexVolume drivers should implement a CSI driver and move users of FlexVolume to CSI. 2232 Users of FlexVolume should move their workloads to CSI driver. 2233 2234 ### Deprecation of klog specific flags 2235 2236 To simplify the code base, several [logging flags got marked as deprecated](https://kubernetes.io/docs/concepts/cluster-administration/system-logs/#klog) in Kubernetes 1.23. 2237 The code which implements them will be removed in a future release, so users of those need to start replacing the deprecated flags with some alternative solutions. 2238 2239 ### Software Supply Chain SLSA Level 1 Compliance in the Kubernetes Release Process 2240 2241 Kubernetes releases are now generating provenance attestation files describing the staging and release phases of the release process and artifacts are verified as they are handed over from one phase to the next. 2242 This final piece completes the work needed to comply with Level 1 of the [SLSA security framework](https://slsa.dev/) (Supply-chain Levels for Software Artifacts). 2243 2244 ### IPv4/IPv6 Dual-stack Networking graduates to GA 2245 2246 [IPv4/IPv6 dual-stack networking](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/563-dual-stack) graduates to GA. 2247 Since 1.21, Kubernetes clusters are enabled to support dual-stack networking by default. 2248 In 1.23, the `IPv6DualStack` feature gate is removed. 2249 The use of dual-stack networking is not mandatory. 2250 Although clusters are enabled to support dual-stack networking, Pods and Services continue to default to single-stack. 2251 To use dual-stack networking: Kubernetes nodes have routable IPv4/IPv6 network interfaces, a dual-stack capable CNI network plugin is used, Pods are configured to be dual-stack and Services have their `.spec.ipFamilyPolicy` field set to either `PreferDualStack` or `RequireDualStack`. 2252 2253 ### HorizontalPodAutoscaler v2 graduates to GA 2254 2255 Version 2 of the HorizontalPodAutoscaler API graduates to stable in the 1.23 release. The HorizontalPodAutoscaler `autoscaling/v2beta2` API is deprecated in favor of the new `autoscaling/v2` API, which the Kubernetes project recommends for all use cases. 2256 2257 This release does *not* deprecate the v1 HorizontalPodAutoscaler API. 2258 2259 ### Generic Ephemeral Volume feature graduates to GA 2260 2261 The generic ephemeral volume feature moved to GA in 1.23. 2262 This feature allows any existing storage driver that supports dynamic provisioning to be used as an ephemeral volume with the volume’s lifecycle bound to the Pod. 2263 All StorageClass parameters for volume provisioning and all features supported with PersistentVolumeClaims are supported. 2264 2265 ### Skip Volume Ownership change graduates to GA 2266 2267 The feature to configure volume permission and ownership change policy for Pods moved to GA in 1.23. 2268 This allows users to skip recursive permission changes on mount and speeds up the pod start up time. 2269 2270 ### Allow CSI drivers to opt-in to volume ownership and permission change graduates to GA 2271 2272 The feature to allow CSI Drivers to declare support for fsGroup based permissions graduates to GA in 1.23. 2273 2274 ### PodSecurity graduates to Beta 2275 2276 [PodSecurity](https://kubernetes.io/docs/concepts/security/pod-security-admission/) moves to Beta. 2277 `PodSecurity` replaces the deprecated `PodSecurityPolicy` admission controller. 2278 `PodSecurity` is an admission controller that enforces Pod Security Standards on Pods in a Namespace based on specific namespace labels that set the enforcement level. 2279 In 1.23, the `PodSecurity` feature gate is enabled by default. 2280 2281 ### Container Runtime Interface (CRI) v1 is default 2282 2283 The Kubelet now supports the CRI `v1` API, which is now the project-wide default. 2284 If a container runtime does not support the `v1` API, Kubernetes will fall back to the `v1alpha2` implementation. 2285 There is no intermediate action required by end-users, because `v1` and `v1alpha2` do not differ in their implementation. 2286 It is likely that `v1alpha2` will be removed in one of the future Kubernetes releases to be able to develop `v1`. 2287 2288 ### Structured logging graduate to Beta 2289 2290 Structured logging reached its Beta milestone. Most log messages from kubelet and kube-scheduler have been converted. Users are encouraged to try out JSON output or parsing of the structured text format and provide feedback on possible solutions for the open issues, such as handling of multi-line strings in log values. 2291 2292 ### Simplified Multi-point plugin configuration for scheduler 2293 2294 The kube-scheduler is adding a new, simplified config field for Plugins to allow multiple extension points to be enabled in one spot. 2295 The new `multiPoint` plugin field is intended to simplify most scheduler setups for administrators. 2296 Plugins that are enabled via `multiPoint` will automatically be registered for each individual extension point that they implement. 2297 For example, a plugin that implements Score and Filter extensions can be simultaneously enabled for both. 2298 This means entire plugins can be enabled and disabled without having to manually edit individual extension point settings. 2299 These extension points can now be abstracted away due to their irrelevance for most users. 2300 2301 ### CSI Migration updates 2302 2303 CSI Migration enables the replacement of existing in-tree storage plugins such as `kubernetes.io/gce-pd` or `kubernetes.io/aws-ebs` with a corresponding CSI driver. 2304 If CSI Migration is working properly, Kubernetes end users shouldn’t notice a difference. 2305 After migration, Kubernetes users may continue to rely on all the functionality of in-tree storage plugins using the existing interface. 2306 - CSI Migration feature is turned on by default but stays in Beta for GCE PD, AWS EBS, and Azure Disk in 1.23. 2307 - CSI Migration is introduced as an Alpha feature for Ceph RBD and Portworx in 1.23. 2308 2309 ## Urgent Upgrade Notes 2310 2311 ### (No, really, you MUST read this before you upgrade) 2312 2313 - Kubeadm: remove the deprecated flag `--experimental-patches` for the `init|join|upgrade` commands. The flag `--patches` is no longer allowed in a mixture with the flag `--config`. Please use the kubeadm configuration for setting patches for a node using `{Init|Join}Configuration.patches`. ([#104065](https://github.com/kubernetes/kubernetes/pull/104065), [@pacoxu](https://github.com/pacoxu)) 2314 - Log messages in JSON format are written to stderr by default now (same as text format) instead of stdout. Users who expected JSON output on stdout must now capture stderr instead or in addition to stdout. ([#106146](https://github.com/kubernetes/kubernetes/pull/106146), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Cluster Lifecycle and Instrumentation] 2315 - Support for the seccomp annotations `seccomp.security.alpha.kubernetes.io/pod` and `container.seccomp.security.alpha.kubernetes.io/[name]` has been deprecated since 1.19, will be dropped in 1.25. Transition to using the `seccompProfile` API field. ([#104389](https://github.com/kubernetes/kubernetes/pull/104389), [@saschagrunert](https://github.com/saschagrunert)) 2316 - [kube-log-runner](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/component-base/logs/kube-log-runner) is included in release tar balls. It can be used to replace the deprecated `--log-file` parameter. ([#106123](https://github.com/kubernetes/kubernetes/pull/106123), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Cloud Provider, Cluster Lifecycle and Instrumentation] 2317 - Kubernetes is built using golang 1.17. This version of go removes the ability to use a `GODEBUG=x509ignoreCN=0` environment setting to re-enable deprecated legacy behavior of treating the CommonName of X.509 serving certificates as a host name. This behavior has been disabled by default since Kubernetes 1.19 / go 1.15. Serving certificates used by admission webhooks, custom resource conversion webhooks, and aggregated API servers must now include valid Subject Alternative Names. If you are running Kubernetes 1.22 with `GODEBUG=x509ignoreCN=0` set, check the `apiserver_kube_aggregator_x509_missing_san_total` and `apiserver_webhooks_x509_missing_san_total` metrics for non-zero values to see if the API server is connecting to webhooks or aggregated API servers using certificates that will be considered invalid in Kubernetes 1.23+. 2318 2319 ## Known Issues 2320 2321 ### Etcd v3.5.[0-2] data corruption 2322 2323 Data corruption issue was found in etcd v3.5.0 release that was shipped with 1.22 Kubernetes release. Please read up-to-date [production recommendations for etcd](https://github.com/etcd-io/etcd/tree/main/CHANGELOG). 2324 2325 ## Changes by Kind 2326 2327 ### Deprecation 2328 2329 - A deprecation notice has been added when using the kube-proxy userspace proxier, which will be removed in v1.25. (#103860) ([#104631](https://github.com/kubernetes/kubernetes/pull/104631), [@perithompson](https://github.com/perithompson)) 2330 - Added `apiserver_longrunning_requests` metric to replace the soon to be deprecated `apiserver_longrunning_gauge` metric. ([#103799](https://github.com/kubernetes/kubernetes/pull/103799), [@jyz0309](https://github.com/jyz0309)) 2331 - Controller-manager: the following flags have no effect and would be removed in v1.24: 2332 - `--port` 2333 - `--address` 2334 The insecure port flags `--port` may only be set to 0 now. 2335 - Kube-scheduler: the `--port` and `--address` flags have no effect and would be removed in v1.24. 2336 The insecure port flags `--port` may only be set to 0 now. 2337 Also `metricsBindAddress` and `healthzBindAddress` fields from `kubescheduler.config.k8s.io/v1beta1` are no-op and expected to be empty. Removed in `kubescheduler.config.k8s.io/v1beta2` completely. ([#96345](https://github.com/kubernetes/kubernetes/pull/96345), [@ingvagabund](https://github.com/ingvagabund)) 2338 In addition, please be careful that: 2339 - kube-scheduler MUST start with `--authorization-kubeconfig` and `--authentication-kubeconfig` correctly set to get authentication/authorization working. 2340 - liveness/readiness probes to kube-scheduler MUST use HTTPS now, and the default port has been changed to 10259. 2341 - Applications that fetch metrics from kube-scheduler should use a dedicated service account which is allowed to access nonResourceURLs `/metrics`. ([#96345](https://github.com/kubernetes/kubernetes/pull/96345), [@ingvagabund](https://github.com/ingvagabund)) [SIG Cloud Provider, Scheduling and Testing] 2342 - Feature-gate `VolumeSubpath` has been deprecated and cannot be disabled. It will be completely removed in 1.25 ([#105474](https://github.com/kubernetes/kubernetes/pull/105474), [@mauriciopoppe](https://github.com/mauriciopoppe)) 2343 - Kubeadm: add a new `output/v1alpha2` API that is identical to the `output/v1alpha1`, but attempts to resolve some internal dependencies with the `kubeadm/v1beta2` API. The `output/v1alpha1` API is now deprecated and will be removed in a future release. ([#105295](https://github.com/kubernetes/kubernetes/pull/105295), [@neolit123](https://github.com/neolit123)) 2344 - Kubeadm: add the kubeadm specific, Alpha (disabled by default) feature gate UnversionedKubeletConfigMap. When this feature is enabled kubeadm will start using a new naming format for the ConfigMap where it stores the KubeletConfiguration structure. The old format included the Kubernetes version - "kube-system/kubelet-config-1.22", while the new format does not - "kube-system/kubelet-config". A similar formatting change is done for the related RBAC rules. The old format is now DEPRECATED and will be removed after the feature graduates to GA. When writing the ConfigMap kubeadm (init, upgrade apply) will respect the value of UnversionedKubeletConfigMap, while when reading it (join, reset, upgrade), it would attempt to use new format first and fallback to the legacy format if needed. ([#105741](https://github.com/kubernetes/kubernetes/pull/105741), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle and Testing] 2345 - Kubeadm: remove the deprecated / NO-OP phase `update-cluster-status` in `kubeadm reset` ([#105888](https://github.com/kubernetes/kubernetes/pull/105888), [@neolit123](https://github.com/neolit123)) 2346 - Remove 'master' as a valid EgressSelection type in the EgressSelectorConfiguration API. ([#102242](https://github.com/kubernetes/kubernetes/pull/102242), [@pacoxu](https://github.com/pacoxu)) 2347 - Removed `kubectl --dry-run` empty default value and boolean values. `kubectl --dry-run` usage must be specified with `--dry-run=(server|client|none)`. ([#105327](https://github.com/kubernetes/kubernetes/pull/105327), [@julianvmodesto](https://github.com/julianvmodesto)) 2348 - Removed deprecated metric `scheduler_volume_scheduling_duration_seconds`. ([#104518](https://github.com/kubernetes/kubernetes/pull/104518), [@dntosas](https://github.com/dntosas)) 2349 - The deprecated `--experimental-bootstrap-kubeconfig` flag has been removed. 2350 This can be set via `--bootstrap-kubeconfig`. ([#103172](https://github.com/kubernetes/kubernetes/pull/103172), [@niulechuan](https://github.com/niulechuan)) 2351 2352 ### API Change 2353 2354 - A new field `omitManagedFields` has been added to both `audit.Policy` and `audit.PolicyRule` 2355 so cluster operators can opt in to omit managed fields of the request and response bodies from 2356 being written to the API audit log. ([#94986](https://github.com/kubernetes/kubernetes/pull/94986), [@tkashem](https://github.com/tkashem)) [SIG API Machinery, Auth, Cloud Provider and Testing] 2357 - A small regression in Service updates was fixed. The circumstances are so unlikely that probably nobody would ever hit it. ([#104601](https://github.com/kubernetes/kubernetes/pull/104601), [@thockin](https://github.com/thockin)) 2358 - Added a feature gate `StatefulSetAutoDeletePVC`, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. ([#99728](https://github.com/kubernetes/kubernetes/pull/99728), [@mattcary](https://github.com/mattcary)) 2359 - Client-go impersonation config can specify a UID to pass impersonated uid information through in requests. ([#104483](https://github.com/kubernetes/kubernetes/pull/104483), [@margocrawf](https://github.com/margocrawf)) 2360 - Create HPA v2 from v2beta2 with some fields changed. ([#102534](https://github.com/kubernetes/kubernetes/pull/102534), [@wangyysde](https://github.com/wangyysde)) [SIG API Machinery, Apps, Auth, Autoscaling and Testing] 2361 - Ephemeral containers graduated to beta and are now available by default. ([#105405](https://github.com/kubernetes/kubernetes/pull/105405), [@verb](https://github.com/verb)) 2362 - Fix kube-proxy regression on UDP services because the logic to detect stale connections was not considering if the endpoint was ready. ([#106163](https://github.com/kubernetes/kubernetes/pull/106163), [@aojea](https://github.com/aojea)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] 2363 - If a conflict occurs when creating an object with `generateName`, the server now returns an "AlreadyExists" error with a retry option. ([#104699](https://github.com/kubernetes/kubernetes/pull/104699), [@vincepri](https://github.com/vincepri)) 2364 - Implement support for recovering from volume expansion failures ([#106154](https://github.com/kubernetes/kubernetes/pull/106154), [@gnufied](https://github.com/gnufied)) [SIG API Machinery, Apps and Storage] 2365 - In kubelet, log verbosity and flush frequency can also be configured via the configuration file and not just via command line flags. In other commands (kube-apiserver, kube-controller-manager), the flags are listed in the "Logs flags" group and not under "Global" or "Misc". The type for `-vmodule` was made a bit more descriptive (`pattern=N,...` instead of `moduleSpec`). ([#106090](https://github.com/kubernetes/kubernetes/pull/106090), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling] 2366 - Introduce `OS` field in the PodSpec ([#104693](https://github.com/kubernetes/kubernetes/pull/104693), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) 2367 - Introduce `v1beta3` API for scheduler. This version 2368 - increases the weight of user specifiable priorities. 2369 The weights of following priority plugins are increased 2370 - `TaintTolerations` to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practice 2371 - `NodeAffinity` to 2 2372 - `InterPodAffinity` to 2 2373 2374 - Won't have `HealthzBindAddress`, `MetricsBindAddress` fields ([#104251](https://github.com/kubernetes/kubernetes/pull/104251), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) 2375 - Introduce v1beta2 for Priority and Fairness with no changes in API spec. ([#104399](https://github.com/kubernetes/kubernetes/pull/104399), [@tkashem](https://github.com/tkashem)) 2376 - JSON log output is configurable and now supports writing info messages to stdout and error messages to stderr. Info messages can be buffered in memory. The default is to write both to stdout without buffering, as before. ([#104873](https://github.com/kubernetes/kubernetes/pull/104873), [@pohly](https://github.com/pohly)) 2377 - JobTrackingWithFinalizers graduates to beta. Feature is enabled by default. ([#105687](https://github.com/kubernetes/kubernetes/pull/105687), [@alculquicondor](https://github.com/alculquicondor)) 2378 - Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums. ([#104969](https://github.com/kubernetes/kubernetes/pull/104969), [@liggitt](https://github.com/liggitt)) 2379 - Kube-apiserver: The `rbac.authorization.k8s.io/v1alpha1` API version is removed; use the `rbac.authorization.k8s.io/v1` API, available since v1.8. The `scheduling.k8s.io/v1alpha1` API version is removed; use the `scheduling.k8s.io/v1` API, available since v1.14. ([#104248](https://github.com/kubernetes/kubernetes/pull/104248), [@liggitt](https://github.com/liggitt)) 2380 - Kube-scheduler: support for configuration file version `v1beta1` is removed. Update configuration files to v1beta2(xref: https://github.com/kubernetes/enhancements/issues/2901) or v1beta3 before upgrading to 1.23. ([#104782](https://github.com/kubernetes/kubernetes/pull/104782), [@kerthcet](https://github.com/kerthcet)) 2381 - KubeSchedulerConfiguration provides a new field `MultiPoint` which will register a plugin for all valid extension points ([#105611](https://github.com/kubernetes/kubernetes/pull/105611), [@damemi](https://github.com/damemi)) [SIG Scheduling and Testing] 2382 - Kubelet should reject pods whose OS doesn't match the node's OS label. ([#105292](https://github.com/kubernetes/kubernetes/pull/105292), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Apps and Node] 2383 - Kubelet: turn the KubeletConfiguration v1beta1 `ResolverConfig` field from a `string` to `*string`. ([#104624](https://github.com/kubernetes/kubernetes/pull/104624), [@Haleygo](https://github.com/Haleygo)) 2384 - Kubernetes is now built using go 1.17. ([#103692](https://github.com/kubernetes/kubernetes/pull/103692), [@justaugustus](https://github.com/justaugustus)) 2385 - Performs strict server side schema validation requests via the `fieldValidation=[Strict,Warn,Ignore]`. ([#105916](https://github.com/kubernetes/kubernetes/pull/105916), [@kevindelgado](https://github.com/kevindelgado)) 2386 - Promote `IPv6DualStack` feature to stable. 2387 Controller Manager flags for the node IPAM controller have slightly changed: 2388 1. When configuring a dual-stack cluster, the user must specify both `--node-cidr-mask-size-ipv4` and `--node-cidr-mask-size-ipv6` to set the per-node IP mask sizes, instead of the previous `--node-cidr-mask-size` flag. 2389 2. The `--node-cidr-mask-size` flag is mutually exclusive with `--node-cidr-mask-size-ipv4` and `--node-cidr-mask-size-ipv6`. 2390 3. Single-stack clusters do not need to change, but may choose to use the more specific flags. Users can use either the older `--node-cidr-mask-size` flag or one of the newer `--node-cidr-mask-size-ipv4` or `--node-cidr-mask-size-ipv6` flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). ([#104691](https://github.com/kubernetes/kubernetes/pull/104691), [@khenidak](https://github.com/khenidak)) 2391 - Remove `NodeLease` feature gate that was graduated and locked to stable in 1.17 release. ([#105222](https://github.com/kubernetes/kubernetes/pull/105222), [@cyclinder](https://github.com/cyclinder)) 2392 - Removed deprecated `--seccomp-profile-root`/`seccompProfileRoot` config. ([#103941](https://github.com/kubernetes/kubernetes/pull/103941), [@saschagrunert](https://github.com/saschagrunert)) 2393 - Since golang 1.17 both net.ParseIP and net.ParseCIDR rejects leading zeros in the dot-decimal notation of IPv4 addresses, 2394 Kubernetes will keep allowing leading zeros on IPv4 address to not break the compatibility. 2395 IMPORTANT: Kubernetes interprets leading zeros on IPv4 addresses as decimal, users must not rely on parser alignment to not being impacted by the associated security advisory: 2396 CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.16.2 and below standard library "net" results in indeterminate SSRF & RFI vulnerabilities. 2397 Reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 ([#104368](https://github.com/kubernetes/kubernetes/pull/104368), [@aojea](https://github.com/aojea)) 2398 - StatefulSet `minReadySeconds` is promoted to beta. ([#104045](https://github.com/kubernetes/kubernetes/pull/104045), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) 2399 - Support pod priority based node graceful shutdown. ([#102915](https://github.com/kubernetes/kubernetes/pull/102915), [@wzshiming](https://github.com/wzshiming)) 2400 - The "Generic Ephemeral Volume" feature graduates to GA. It is now enabled unconditionally. ([#105609](https://github.com/kubernetes/kubernetes/pull/105609), [@pohly](https://github.com/pohly)) 2401 - The Kubelet's `--register-with-taints` option is now available via the Kubelet config file field registerWithTaints ([#105437](https://github.com/kubernetes/kubernetes/pull/105437), [@cmssczy](https://github.com/cmssczy)) [SIG Node and Scalability] 2402 - The `CSIDriver.Spec.StorageCapacity` can now be modified. ([#101789](https://github.com/kubernetes/kubernetes/pull/101789), [@pohly](https://github.com/pohly)) 2403 - The `CSIVolumeFSGroupPolicy` feature has moved from beta to GA. ([#105940](https://github.com/kubernetes/kubernetes/pull/105940), [@dobsonj](https://github.com/dobsonj)) 2404 - The `IngressClass.Spec.Parameters.Namespace` field is now GA. ([#104636](https://github.com/kubernetes/kubernetes/pull/104636), [@hbagdi](https://github.com/hbagdi)) 2405 - The `Service.spec.ipFamilyPolicy` field is now *required* in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either `ipFamilies` or `clusterIPs`, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify `ipFamilyPolicy` as either "PreferDualStack" or "RequireDualStack". ([#96684](https://github.com/kubernetes/kubernetes/pull/96684), [@thockin](https://github.com/thockin)) 2406 - The `TTLAfterFinished` feature gate is now GA and enabled by default. ([#105219](https://github.com/kubernetes/kubernetes/pull/105219), [@sahilvv](https://github.com/sahilvv)) 2407 - The `kube-controller-manager` supports `--concurrent-ephemeralvolume-syncs` flag to set the number of ephemeral volume controller workers. ([#102981](https://github.com/kubernetes/kubernetes/pull/102981), [@SataQiu](https://github.com/SataQiu)) 2408 - The legacy scheduler policy config is removed in v1.23, the associated flags `policy-config-file`, `policy-configmap`, `policy-configmap-namespace` and `use-legacy-policy-config` are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. ([#105424](https://github.com/kubernetes/kubernetes/pull/105424), [@kerthcet](https://github.com/kerthcet)) 2409 - Track the number of Pods with a Ready condition in Job status. The feature is alpha and needs the feature gate JobReadyPods to be enabled. ([#104915](https://github.com/kubernetes/kubernetes/pull/104915), [@alculquicondor](https://github.com/alculquicondor)) 2410 - Users of `LogFormatRegistry` in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no `v` field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). ([#104103](https://github.com/kubernetes/kubernetes/pull/104103), [@pohly](https://github.com/pohly)) 2411 - Validation rules for Custom Resource Definitions can be written in the [CEL expression language](https://github.com/google/cel-spec) using the `x-kubernetes-validations` extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. ([#106051](https://github.com/kubernetes/kubernetes/pull/106051), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] 2412 2413 ### Feature 2414 2415 - (beta feature) If the CSI driver supports the NodeServiceCapability `VOLUME_MOUNT_GROUP` and the `DelegateFSGroupToCSIDriver` feature gate is enabled, kubelet will delegate applying FSGroup to the driver by passing it to NodeStageVolume and NodePublishVolume, regardless of what other FSGroup policies are set. ([#106330](https://github.com/kubernetes/kubernetes/pull/106330), [@verult](https://github.com/verult)) [SIG Storage] 2416 - Add a new `distribute-cpus-across-numa` option to the static `CPUManager` policy. When enabled, this will trigger the `CPUManager` to evenly distribute CPUs across NUMA nodes in cases where more than one NUMA node is required to satisfy the allocation. ([#105631](https://github.com/kubernetes/kubernetes/pull/105631), [@klueska](https://github.com/klueska)) 2417 - Add fish shell completion to kubectl. ([#92989](https://github.com/kubernetes/kubernetes/pull/92989), [@WLun001](https://github.com/WLun001)) 2418 - Add mechanism to load simple sniffer class into fluentd-elasticsearch image ([#92853](https://github.com/kubernetes/kubernetes/pull/92853), [@cosmo0920](https://github.com/cosmo0920)) 2419 - Add support for Portworx plugin to csi-translation-lib. Alpha release 2420 2421 Portworx CSI driver is required to enable migration. 2422 This PR adds support of the `CSIMigrationPortworx` feature gate, which can be enabled by: 2423 2424 1. Adding the feature flag to the kube-controller-manager `--feature-gates=CSIMigrationPortworx=true` 2425 2. Adding the feature flag to the kubelet config: 2426 2427 featureGates: 2428 CSIMigrationPortworx: true ([#103447](https://github.com/kubernetes/kubernetes/pull/103447), [@trierra](https://github.com/trierra)) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] 2429 - Add support to generate client-side binaries for windows/arm64 platform ([#104894](https://github.com/kubernetes/kubernetes/pull/104894), [@pacoxu](https://github.com/pacoxu)) 2430 - Added PowerShell completion generation by running `kubectl completion powershell`. ([#103758](https://github.com/kubernetes/kubernetes/pull/103758), [@zikhan](https://github.com/zikhan)) 2431 - Added a `Processing` condition for the `workqueue` API. 2432 Changed `Shutdown` for the `workqueue` API to wait until the work queue finishes processing all in-flight items. ([#101928](https://github.com/kubernetes/kubernetes/pull/101928), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) 2433 - Added a new feature gate `CustomResourceValidationExpressions` to enable expression validation for Custom Resource. ([#105107](https://github.com/kubernetes/kubernetes/pull/105107), [@cici37](https://github.com/cici37)) 2434 - Added a new flag `--append-server-path` to `kubectl proxy` that will automatically append the kube context server path to each request. ([#97350](https://github.com/kubernetes/kubernetes/pull/97350), [@FabianKramm](https://github.com/FabianKramm)) 2435 - Added ability for `kubectl wait` to wait on arbitary JSON path ([#105776](https://github.com/kubernetes/kubernetes/pull/105776), [@lauchokyip](https://github.com/lauchokyip)) 2436 - Added support for `PodAndContainerStatsFromCRI` feature gate, which allows a user to specify their pod stats must also come from the CRI, not `cAdvisor`. ([#103095](https://github.com/kubernetes/kubernetes/pull/103095), [@haircommander](https://github.com/haircommander)) 2437 - Added support for setting controller-manager log level online. ([#104571](https://github.com/kubernetes/kubernetes/pull/104571), [@h4ghhh](https://github.com/h4ghhh)) 2438 - Added the ability to specify whether to use an RFC7396 JSON Merge Patch, an RFC6902 JSON Patch, or a Strategic Merge Patch to perform an override of the resources created by `kubectl run` and `kubectl expose`. ([#105140](https://github.com/kubernetes/kubernetes/pull/105140), [@brianpursley](https://github.com/brianpursley)) 2439 - Adding option for `kubectl cp` to resume on network errors until completion, requires tar in addition to tail inside the container image ([#104792](https://github.com/kubernetes/kubernetes/pull/104792), [@matthyx](https://github.com/matthyx)) 2440 - Adding support for multiple `--from-env-file flags`. ([#104232](https://github.com/kubernetes/kubernetes/pull/104232), [@lauchokyip](https://github.com/lauchokyip)) 2441 - Adding support for multiple `--from-env-file` flags. ([#101646](https://github.com/kubernetes/kubernetes/pull/101646), [@lauchokyip](https://github.com/lauchokyip)) 2442 - Adds `--as-uid` flag to `kubectl` to allow uid impersonation in the same way as user and group impersonation. ([#105794](https://github.com/kubernetes/kubernetes/pull/105794), [@margocrawf](https://github.com/margocrawf)) 2443 - Adds new [alpha] command 'kubectl events' ([#99557](https://github.com/kubernetes/kubernetes/pull/99557), [@bboreham](https://github.com/bboreham)) 2444 - Allow node expansion of local volumes. ([#102886](https://github.com/kubernetes/kubernetes/pull/102886), [@gnufied](https://github.com/gnufied)) 2445 - Allow to build kubernetes with a custom `kube-cross` image. ([#104185](https://github.com/kubernetes/kubernetes/pull/104185), [@dims](https://github.com/dims)) 2446 - Allows users to prevent garbage collection on pinned images ([#103299](https://github.com/kubernetes/kubernetes/pull/103299), [@wgahnagl](https://github.com/wgahnagl)) [SIG Node] 2447 - CRI v1 is now the project default. If a container runtime does not support the v1 API, Kubernetes will fall back to the v1alpha2 implementation. ([#106501](https://github.com/kubernetes/kubernetes/pull/106501), [@ehashman](https://github.com/ehashman)) 2448 - Changed feature `CSIMigrationAWS` to on by default. This feature requires the AWS EBS CSI driver to be installed. ([#106098](https://github.com/kubernetes/kubernetes/pull/106098), [@wongma7](https://github.com/wongma7)) 2449 - Client-go: pass `DeleteOptions` down to the fake client `Reactor` ([#102945](https://github.com/kubernetes/kubernetes/pull/102945), [@chenchun](https://github.com/chenchun)) 2450 - Cloud providers can set service account names for cloud controllers. ([#103178](https://github.com/kubernetes/kubernetes/pull/103178), [@nckturner](https://github.com/nckturner)) 2451 - Display Labels when kubectl describe ingress. ([#103894](https://github.com/kubernetes/kubernetes/pull/103894), [@kabab](https://github.com/kabab)) 2452 - Enhance scheduler `VolumeBinding` plugin to handle Lost PVC as `UnschedulableAndUnresolvable` ([#105245](https://github.com/kubernetes/kubernetes/pull/105245), [@yibozhuang](https://github.com/yibozhuang)) 2453 - Ensures that volume is deleted from the storage backend when the user tries to delete the PV object manually and the PV `ReclaimPolicy` is set to `Delete`. ([#105773](https://github.com/kubernetes/kubernetes/pull/105773), [@deepakkinni](https://github.com/deepakkinni)) 2454 - Expose a `NewUnstructuredExtractor` from apply configurations `meta/v1` package that enables extracting objects into unstructured apply configurations. ([#103564](https://github.com/kubernetes/kubernetes/pull/103564), [@kevindelgado](https://github.com/kevindelgado)) 2455 - Feature gate `StorageObjectInUseProtection` has been deprecated and cannot be disabled. It will be completely removed in 1.25 ([#105495](https://github.com/kubernetes/kubernetes/pull/105495), [@ikeeip](https://github.com/ikeeip)) 2456 - Graduating `controller_admission_duration_seconds`, `step_admission_duration_seconds`, `webhook_admission_duration_seconds`, `apiserver_current_inflight_requests` and `apiserver_response_sizes` metrics to stable. ([#106122](https://github.com/kubernetes/kubernetes/pull/106122), [@rezakrimi](https://github.com/rezakrimi)) [SIG API Machinery, Instrumentation and Testing] 2457 - Graduating `pending_pods`, `preemption_attempts_total`, `preemption_victims` and `schedule_attempts_total` metrics to stable. Also `e2e_scheduling_duration_seconds` is renamed to `scheduling_attempt_duration_seconds` and the latter is graduated to stable. ([#105941](https://github.com/kubernetes/kubernetes/pull/105941), [@rezakrimi](https://github.com/rezakrimi)) [SIG Instrumentation, Scheduling and Testing] 2458 - Health check of kube-controller-manager now includes each controller. ([#104667](https://github.com/kubernetes/kubernetes/pull/104667), [@jiahuif](https://github.com/jiahuif)) 2459 - Integration testing now takes periodic Prometheus scrapes from the etcd server. 2460 There is a new script ,`hack/run-prometheus-on-etcd-scrapes.sh`, that runs a containerized Prometheus server against an archive of such scrapes. ([#106190](https://github.com/kubernetes/kubernetes/pull/106190), [@MikeSpreitzer](https://github.com/MikeSpreitzer)) [SIG API Machinery and Testing] 2461 - Introduce a feature gate `DisableKubeletCloudCredentialProviders` which allows disabling the in-tree kubelet credential providers. 2462 2463 The feature gate `DisableKubeletCloudCredentialProviders` is currently in Alpha, which means is currently disabled by default. Once this feature gate moves to beta, in-tree credential providers will be disabled by default, and users will need to migrate to use external credential providers. ([#102507](https://github.com/kubernetes/kubernetes/pull/102507), [@ostrain](https://github.com/ostrain)) 2464 - Introduces a new metric: `admission_webhook_request_total` with the following labels: name (string) - the webhook name, type (string) - the admission type, operation (string) - the requested verb, code (int) - the HTTP status code, rejected (bool) - whether the request was rejected, namespace (string) - the namespace of the requested resource. ([#103162](https://github.com/kubernetes/kubernetes/pull/103162), [@rmoriar1](https://github.com/rmoriar1)) 2465 - Kubeadm: add support for dry running `kubeadm join`. The new flag `kubeadm join --dry-run` is similar to the existing flag for `kubeadm init/upgrade` and allows you to see what changes would be applied. ([#103027](https://github.com/kubernetes/kubernetes/pull/103027), [@Haleygo](https://github.com/Haleygo)) 2466 - Kubeadm: do not check if the `/etc/kubernetes/manifests` folder is empty on joining worker nodes during preflight ([#104942](https://github.com/kubernetes/kubernetes/pull/104942), [@SataQiu](https://github.com/SataQiu)) 2467 - Kubectl will now provide shell completion choices for the `--output/-o` flag ([#105851](https://github.com/kubernetes/kubernetes/pull/105851), [@marckhouzam](https://github.com/marckhouzam)) 2468 - Kubelet should reconcile `kubernetes.io/os` and `kubernetes.io/arch` labels on the node object. The side-effect of this is kubelet would deny admission to pod which has nodeSelector with label `kubernetes.io/os` or `kubernetes.io/arch` which doesn't match the underlying OS or arch on the host OS. 2469 - The label reconciliation happens as part of periodic status update which can be configured via flag `--node-status-update-frequency` ([#104613](https://github.com/kubernetes/kubernetes/pull/104613), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Node, Testing and Windows] 2470 - Kubernetes is now built with Golang 1.16.7. ([#104199](https://github.com/kubernetes/kubernetes/pull/104199), [@cpanato](https://github.com/cpanato)) 2471 - Kubernetes is now built with Golang 1.17.1. ([#104904](https://github.com/kubernetes/kubernetes/pull/104904), [@cpanato](https://github.com/cpanato)) 2472 - Kubernetes is now built with Golang 1.17.2 ([#105563](https://github.com/kubernetes/kubernetes/pull/105563), [@mengjiao-liu](https://github.com/mengjiao-liu)) 2473 - Kubernetes is now built with Golang 1.17.3 ([#106209](https://github.com/kubernetes/kubernetes/pull/106209), [@cpanato](https://github.com/cpanato)) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] 2474 - Move `ConfigurableFSGroupPolicy` to GA and rename metric `volume_fsgroup_recursive_apply` to `volume_apply_access_control` ([#105885](https://github.com/kubernetes/kubernetes/pull/105885), [@gnufied](https://github.com/gnufied)) 2475 - Move the `GetAllocatableResources` Endpoint in PodResource API to the beta that will make it enabled by default. ([#105003](https://github.com/kubernetes/kubernetes/pull/105003), [@swatisehgal](https://github.com/swatisehgal)) 2476 - Moving `WindowsHostProcessContainers` feature to beta ([#106058](https://github.com/kubernetes/kubernetes/pull/106058), [@marosset](https://github.com/marosset)) 2477 - Node affinity, Node selectors, and tolerations are now mutable for Jobs that are suspended and have never been started ([#105479](https://github.com/kubernetes/kubernetes/pull/105479), [@ahg-g](https://github.com/ahg-g)) 2478 - Pod template annotations and labels are now mutable for Jobs that are suspended and have never been started ([#105980](https://github.com/kubernetes/kubernetes/pull/105980), [@ahg-g](https://github.com/ahg-g)) 2479 - PodSecurity: in 1.23+ restricted policy levels, Pods and containers which set `runAsUser=0` are forbidden at admission-time; previously, they would be rejected at runtime ([#105857](https://github.com/kubernetes/kubernetes/pull/105857), [@liggitt](https://github.com/liggitt)) 2480 - Shell completion now knows to continue suggesting resource names when the command supports it. For example `kubectl get pod pod1 <TAB>` will suggest more Pod names. ([#105711](https://github.com/kubernetes/kubernetes/pull/105711), [@marckhouzam](https://github.com/marckhouzam)) 2481 - Support to enable Hyper-V in GCE Windows Nodes created with `kube-up` ([#105999](https://github.com/kubernetes/kubernetes/pull/105999), [@mauriciopoppe](https://github.com/mauriciopoppe)) 2482 - The CPUManager policy options are now enabled, and we introduce a graduation path for the new CPU Manager policy options. ([#105012](https://github.com/kubernetes/kubernetes/pull/105012), [@fromanirh](https://github.com/fromanirh)) 2483 - The Pods and Pod controllers that are exempted from the PodSecurity admission process are now marked with the `pod-security.kubernetes.io/exempt: user/namespace/runtimeClass` annotation, based on what caused the exemption. 2484 2485 The enforcement level that allowed or denied a Pod during PodSecurity admission is now marked by the `pod-security.kubernetes.io/enforce-policy` annotation. 2486 2487 The annotation that informs about audit policy violations changed from `pod-security.kubernetes.io/audit` to `pod-security.kubernetes.io/audit-violation`. ([#105908](https://github.com/kubernetes/kubernetes/pull/105908), [@stlaz](https://github.com/stlaz)) 2488 - The `/openapi/v3` endpoint will be populated with OpenAPI v3 if the feature flag is enabled ([#105945](https://github.com/kubernetes/kubernetes/pull/105945), [@Jefftree](https://github.com/Jefftree)) 2489 - The `CSIMigrationGCE` feature flag is turned `ON` by default ([#104722](https://github.com/kubernetes/kubernetes/pull/104722), [@leiyiz](https://github.com/leiyiz)) 2490 - The `DownwardAPIHugePages` feature is now enabled by default. ([#106271](https://github.com/kubernetes/kubernetes/pull/106271), [@mysunshine92](https://github.com/mysunshine92)) 2491 - The `PodSecurity `admission plugin has graduated to `beta` and is enabled by default. The admission configuration version has been promoted to `pod-security.admission.config.k8s.io/v1beta1`. See https://kubernetes.io/docs/concepts/security/pod-security-admission/ for usage guidelines. ([#106089](https://github.com/kubernetes/kubernetes/pull/106089), [@liggitt](https://github.com/liggitt)) 2492 - The `ServiceAccountIssuerDiscovery` feature gate is removed. It reached GA in Kubernetes 1.21. ([#103685](https://github.com/kubernetes/kubernetes/pull/103685), [@mengjiao-liu](https://github.com/mengjiao-liu)) 2493 - The `constants/variables` from k8s.io for STABLE metrics is now supported. ([#103654](https://github.com/kubernetes/kubernetes/pull/103654), [@coffeepac](https://github.com/coffeepac)) 2494 - The `kubectl describe namespace` now shows Conditions ([#106219](https://github.com/kubernetes/kubernetes/pull/106219), [@dlipovetsky](https://github.com/dlipovetsky)) 2495 - The etcd container image now supports Windows. ([#92433](https://github.com/kubernetes/kubernetes/pull/92433), [@claudiubelu](https://github.com/claudiubelu)) 2496 - The kube-apiserver's Prometheus metrics have been extended with some that describe the costs of handling LIST requests. They are as follows. 2497 - *apiserver_cache_list_total*: Counter of LIST requests served from watch cache, broken down by resource_prefix and index_name 2498 - *apiserver_cache_list_fetched_objects_total*: Counter of objects read from watch cache in the course of serving a LIST request, broken down by resource_prefix and index_name 2499 - *apiserver_cache_list_evaluated_objects_total*: Counter of objects tested in the course of serving a LIST request from watch cache, broken down by resource_prefix 2500 - *apiserver_cache_list_returned_objects_total*: Counter of objects returned for a LIST request from watch cache, broken down by resource_prefix 2501 - *apiserver_storage_list_total*: Counter of LIST requests served from etcd, broken down by resource 2502 - *apiserver_storage_list_fetched_objects_total*: Counter of objects read from etcd in the course of serving a LIST request, broken down by resource 2503 - *apiserver_storage_list_evaluated_objects_total*: Counter of objects tested in the course of serving a LIST request from etcd, broken down by resource 2504 - *apiserver_storage_list_returned_objects_total*: Counter of objects returned for a LIST request from etcd, broken down by resource ([#104983](https://github.com/kubernetes/kubernetes/pull/104983), [@MikeSpreitzer](https://github.com/MikeSpreitzer)) 2505 - The pause image list now contains Windows Server 2022. ([#104438](https://github.com/kubernetes/kubernetes/pull/104438), [@nick5616](https://github.com/nick5616)) 2506 - The script `kube-up.sh` installs `csi-proxy v1.0.1-gke.0`. ([#104426](https://github.com/kubernetes/kubernetes/pull/104426), [@mauriciopoppe](https://github.com/mauriciopoppe)) 2507 - This PR adds the following metrics for API Priority and Fairness. 2508 - **apiserver_flowcontrol_priority_level_seat_count_samples**: histograms of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level; the observations are taken once per millisecond. 2509 - **apiserver_flowcontrol_priority_level_seat_count_watermarks**: histograms of high and low watermarks of number of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level. 2510 - **apiserver_flowcontrol_watch_count_samples**: histograms of number of watches relevant to a given mutating request, broken down by that request's priority_level and flow_schema. ([#105873](https://github.com/kubernetes/kubernetes/pull/105873), [@MikeSpreitzer](https://github.com/MikeSpreitzer)) [SIG API Machinery, Instrumentation and Testing] 2511 - Topology Aware Hints have graduated to beta. ([#106433](https://github.com/kubernetes/kubernetes/pull/106433), [@robscott](https://github.com/robscott)) [SIG Network] 2512 - Turn on CSIMigrationAzureDisk by default on 1.23 ([#104670](https://github.com/kubernetes/kubernetes/pull/104670), [@andyzhangx](https://github.com/andyzhangx)) 2513 - Update the system-validators library to v1.6.0 ([#106323](https://github.com/kubernetes/kubernetes/pull/106323), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle and Node] 2514 - Updated Cluster Autosaler to version `1.22.0`. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.22.0. ([#104293](https://github.com/kubernetes/kubernetes/pull/104293), [@x13n](https://github.com/x13n)) 2515 - Updates `debian-iptables` to v1.6.7 to pick up CVE fixes. ([#104970](https://github.com/kubernetes/kubernetes/pull/104970), [@PushkarJ](https://github.com/PushkarJ)) 2516 - Updates the following images to pick up CVE fixes: 2517 - `debian` to v1.9.0 2518 - `debian-iptables` to v1.6.6 2519 - `setcap` to v2.0.4 ([#104142](https://github.com/kubernetes/kubernetes/pull/104142), [@mengjiao-liu](https://github.com/mengjiao-liu)) 2520 - Upgrade etcd to 3.5.1 ([#105706](https://github.com/kubernetes/kubernetes/pull/105706), [@uthark](https://github.com/uthark)) [SIG Cloud Provider, Cluster Lifecycle and Testing] 2521 - When feature gate `JobTrackingWithFinalizers` is enabled: 2522 - Limit the number of Pods tracked in a single Job sync to avoid starvation of small Jobs. 2523 - The metric `job_pod_finished_total` counts the number of finished Pods tracked by the Job controller. ([#105197](https://github.com/kubernetes/kubernetes/pull/105197), [@alculquicondor](https://github.com/alculquicondor)) 2524 - When using `RequestedToCapacityRatio` ScoringStrategy, empty shape will cause error. ([#106169](https://github.com/kubernetes/kubernetes/pull/106169), [@kerthcet](https://github.com/kerthcet)) [SIG Scheduling] 2525 - `client-go` event library allows customizing spam filtering function. 2526 It is now possible to override `SpamKeyFunc`, which is used by event filtering to detect spam in the events. ([#103918](https://github.com/kubernetes/kubernetes/pull/103918), [@olagacek](https://github.com/olagacek)) 2527 - `client-go`, using log level 9, traces the following events of a HTTP request: 2528 - DNS lookup 2529 - TCP dialing 2530 - TLS handshake 2531 - Time to get a connection from the pool 2532 - Time to process a request ([#105156](https://github.com/kubernetes/kubernetes/pull/105156), [@aojea](https://github.com/aojea)) 2533 2534 ### Documentation 2535 2536 - Graduating `pod_scheduling_duration_seconds`, `pod_scheduling_attempts`, `framework_extension_point_duration_seconds`, `plugin_execution_duration_seconds` and `queue_incoming_pods_total` metrics to stable. ([#106266](https://github.com/kubernetes/kubernetes/pull/106266), [@ahg-g](https://github.com/ahg-g)) [SIG Instrumentation, Scheduling and Testing] 2537 - The test "[sig-network] EndpointSlice should have Endpoints and EndpointSlices pointing to API Server [Conformance]" only requires that there is an EndpointSlice that references the "kubernetes.default" service, it no longer requires that its named "kubernetes". ([#104664](https://github.com/kubernetes/kubernetes/pull/104664), [@aojea](https://github.com/aojea)) 2538 - Update description of `--audit-log-maxbackup` to describe behavior when `value = 0`. ([#103843](https://github.com/kubernetes/kubernetes/pull/103843), [@Arkessler](https://github.com/Arkessler)) 2539 - Users should not rely on unsupported CRON_TZ variable when specifying schedule, both the API server and cronjob controller will emit warnings pointing to https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/ containing explanation ([#106455](https://github.com/kubernetes/kubernetes/pull/106455), [@soltysh](https://github.com/soltysh)) [SIG Apps] 2540 2541 ### Failing Test 2542 2543 - Fixes hostPath storage E2E tests within SELinux enabled env ([#104551](https://github.com/kubernetes/kubernetes/pull/104551), [@Elbehery](https://github.com/Elbehery)) 2544 2545 ### Bug or Regression 2546 2547 - (PodSecurity admission) errors validating workload resources (deployment, replicaset, etc.) no longer block admission. ([#106017](https://github.com/kubernetes/kubernetes/pull/106017), [@tallclair](https://github.com/tallclair)) [SIG Auth] 2548 - A pod that the Kubelet rejects was still considered as being accepted for a brief period of time after rejection, which might cause some pods to be rejected briefly that could fit on the node. A pod that is still terminating (but has status indicating it has failed) may also still be consuming resources and so should also be considered. ([#104817](https://github.com/kubernetes/kubernetes/pull/104817), [@smarterclayton](https://github.com/smarterclayton)) 2549 - Add Kubernetes Events to the `Kubelet Graceful Shutdown` feature. ([#101081](https://github.com/kubernetes/kubernetes/pull/101081), [@rphillips](https://github.com/rphillips)) 2550 - Add Pod Security admission metrics: `pod_security_evaluations_total`, `pod_security_exemptions_total`, `pod_security_errors_total` ([#105898](https://github.com/kubernetes/kubernetes/pull/105898), [@tallclair](https://github.com/tallclair)) 2551 - Add support for Windows Network stats in Containerd ([#105744](https://github.com/kubernetes/kubernetes/pull/105744), [@jsturtevant](https://github.com/jsturtevant)) [SIG Node, Testing and Windows] 2552 - Added show-capacity option to `kubectl top node` to show `Capacity` resource usage ([#102917](https://github.com/kubernetes/kubernetes/pull/102917), [@bysnupy](https://github.com/bysnupy)) [SIG CLI] 2553 - Apimachinery: Pretty printed JSON and YAML output is now indented consistently. ([#105466](https://github.com/kubernetes/kubernetes/pull/105466), [@liggitt](https://github.com/liggitt)) 2554 - Be able to create a Pod with Generic Ephemeral Volumes as raw block devices. ([#105682](https://github.com/kubernetes/kubernetes/pull/105682), [@pohly](https://github.com/pohly)) 2555 - CA, certificate and key bundles for the `generic-apiserver` based servers will be reloaded immediately after the files are changed. ([#104102](https://github.com/kubernetes/kubernetes/pull/104102), [@tnqn](https://github.com/tnqn)) 2556 - Change `kubectl diff --invalid-arg` status code from 1 to 2 to match docs ([#105445](https://github.com/kubernetes/kubernetes/pull/105445), [@ardaguclu](https://github.com/ardaguclu)) 2557 - Changed kubectl describe to compute age of an event using the `EventSeries.count` and `EventSeries.lastObservedTime`. ([#104482](https://github.com/kubernetes/kubernetes/pull/104482), [@harjas27](https://github.com/harjas27)) 2558 - Changes behaviour of kube-proxy start; does not attempt to set specific `sysctl` values (which does not work in recent Kernel versions anymore in non-init namespaces), when the current sysctl values are already set higher. ([#103174](https://github.com/kubernetes/kubernetes/pull/103174), [@Napsty](https://github.com/Napsty)) 2559 - Client-go uses the same HTTP client for all the generated groups and versions, allowing to share customized transports for multiple groups versions. ([#105490](https://github.com/kubernetes/kubernetes/pull/105490), [@aojea](https://github.com/aojea)) 2560 - Disable aufs module for gce clusters. ([#103831](https://github.com/kubernetes/kubernetes/pull/103831), [@lizhuqi](https://github.com/lizhuqi)) 2561 - Do not unmount and mount subpath bind mounts during container creation unless bind mount changes ([#105512](https://github.com/kubernetes/kubernetes/pull/105512), [@gnufied](https://github.com/gnufied)) [SIG Storage] 2562 - Don't prematurely close reflectors in case of slow initialization in watch based manager to fix issues with inability to properly mount secrets/configmaps. ([#104604](https://github.com/kubernetes/kubernetes/pull/104604), [@wojtek-t](https://github.com/wojtek-t)) 2563 - Don't use a custom dialer for the kubelet if is not rotating certificates, so we can reuse TCP connections and have only one between the apiserver and the kubelet. 2564 If users experiment problems with stale connections using HTTP1.1, they can force the previous behavior of the kubelet by setting the environment variable DISABLE_HTTP2. ([#104844](https://github.com/kubernetes/kubernetes/pull/104844), [@aojea](https://github.com/aojea)) [SIG API Machinery, Auth and Node] 2565 - EndpointSlice Mirroring controller now cleans up managed EndpointSlices when a Service selector is added ([#105997](https://github.com/kubernetes/kubernetes/pull/105997), [@robscott](https://github.com/robscott)) [SIG Apps, Network and Testing] 2566 - Enhanced event messages for pod failed for exec probe timeout ([#106201](https://github.com/kubernetes/kubernetes/pull/106201), [@yxxhero](https://github.com/yxxhero)) [SIG Node] 2567 - Ensure Pods are removed from the scheduler cache when the scheduler misses deletion events due to transient errors ([#106102](https://github.com/kubernetes/kubernetes/pull/106102), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] 2568 - Ensure `InstanceShutdownByProviderID` return false for creating Azure VMs. ([#104382](https://github.com/kubernetes/kubernetes/pull/104382), [@feiskyer](https://github.com/feiskyer)) 2569 - Evicted and other terminated Pods will no longer revert to the `Running` phase. ([#105462](https://github.com/kubernetes/kubernetes/pull/105462), [@ehashman](https://github.com/ehashman)) 2570 - Fix `kube-apiserver` metric reporting for the deprecated watch path of `/api/<version>/watch/...`. ([#104161](https://github.com/kubernetes/kubernetes/pull/104161), [@wojtek-t](https://github.com/wojtek-t)) 2571 - Fix a regression where the Kubelet failed to exclude already completed pods from calculations about how many resources it was currently using when deciding whether to allow more pods. ([#104577](https://github.com/kubernetes/kubernetes/pull/104577), [@smarterclayton](https://github.com/smarterclayton)) 2572 - Fix detach disk issue on deleting vmss node. ([#104572](https://github.com/kubernetes/kubernetes/pull/104572), [@andyzhangx](https://github.com/andyzhangx)) 2573 - Fix job controller syncs: In case of conflicts, ensure that the sync happens with the most up to date information. Improves reliability of JobTrackingWithFinalizers. ([#105214](https://github.com/kubernetes/kubernetes/pull/105214), [@alculquicondor](https://github.com/alculquicondor)) 2574 - Fix job tracking with finalizers for more than 500 pods, ensuring all finalizers are removed before counting the Pod. ([#104666](https://github.com/kubernetes/kubernetes/pull/104666), [@alculquicondor](https://github.com/alculquicondor)) 2575 - Fix pod name of NonIndexed Jobs to not include rogue `-1` substring ([#105676](https://github.com/kubernetes/kubernetes/pull/105676), [@alculquicondor](https://github.com/alculquicondor)) 2576 - Fix scoring for `NodeResourcesBalancedAllocation` plugins when nodes have containers with no requests. ([#105845](https://github.com/kubernetes/kubernetes/pull/105845), [@ahmad-diaa](https://github.com/ahmad-diaa)) 2577 - Fix system default topology spreading when nodes don't have zone labels. Pods correctly spread by default now. ([#105046](https://github.com/kubernetes/kubernetes/pull/105046), [@alculquicondor](https://github.com/alculquicondor)) 2578 - Fix: do not try to delete a LoadBalancer that does not exist ([#105777](https://github.com/kubernetes/kubernetes/pull/105777), [@nilo19](https://github.com/nilo19)) 2579 - Fix: ignore non-VMSS error for VMAS nodes in `reconcileBackendPools`. ([#103997](https://github.com/kubernetes/kubernetes/pull/103997), [@nilo19](https://github.com/nilo19)) 2580 - Fix: leave the probe path empty for TCP probes ([#105253](https://github.com/kubernetes/kubernetes/pull/105253), [@nilo19](https://github.com/nilo19)) 2581 - Fix: remove VMSS and VMSS instances from SLB backend pool only when necessary ([#105839](https://github.com/kubernetes/kubernetes/pull/105839), [@nilo19](https://github.com/nilo19)) 2582 - Fix: skip `instance not found` when decoupling VMSSs from LB ([#105666](https://github.com/kubernetes/kubernetes/pull/105666), [@nilo19](https://github.com/nilo19)) 2583 - Fix: skip case sensitivity when checking Azure NSG rules. ([#104384](https://github.com/kubernetes/kubernetes/pull/104384), [@feiskyer](https://github.com/feiskyer)) 2584 - Fixed a bug that prevents a PersistentVolume that has a PersistentVolumeClaim UID which doesn't exist in local cache but exists in etcd from being updated to the Released phase. ([#105211](https://github.com/kubernetes/kubernetes/pull/105211), [@xiaopingrubyist](https://github.com/xiaopingrubyist)) 2585 - Fixed a bug where using `kubectl patch` with `$deleteFromPrimitiveList` on a nonexistent or empty list would add the item to the list ([#105421](https://github.com/kubernetes/kubernetes/pull/105421), [@brianpursley](https://github.com/brianpursley)) 2586 - Fixed a bug which could cause webhooks to have an incorrect copy of the old object after an Apply or Update ([#106195](https://github.com/kubernetes/kubernetes/pull/106195), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery] 2587 - Fixed a bug which kubectl would emit duplicate warning messages for flag names that contain an underscore and recommend using a nonexistent flag in some cases. ([#103852](https://github.com/kubernetes/kubernetes/pull/103852), [@brianpursley](https://github.com/brianpursley)) 2588 - Fixed a panic in `kubectl` when creating secrets with an improper output type ([#106317](https://github.com/kubernetes/kubernetes/pull/106317), [@lauchokyip](https://github.com/lauchokyip)) 2589 - Fixed a regression setting `--audit-log-path=-` to log to stdout in 1.22 pre-release. ([#103875](https://github.com/kubernetes/kubernetes/pull/103875), [@andrewrynhard](https://github.com/andrewrynhard)) 2590 - Fixed an issue which didn't append OS's environment variables with the one provided in Credential Provider Config file, which may fail execution of external credential provider binary. 2591 See https://github.com/kubernetes/kubernetes/issues/102750. ([#103231](https://github.com/kubernetes/kubernetes/pull/103231), [@n4j](https://github.com/n4j)) 2592 - Fixed applying of SELinux labels to CSI volumes on very busy systems (with "error checking for SELinux support: could not get consistent content of /proc/self/mountinfo after 3 attempts") ([#105934](https://github.com/kubernetes/kubernetes/pull/105934), [@jsafrane](https://github.com/jsafrane)) [SIG Storage] 2593 - Fixed architecture within manifest for non `amd64` etcd images. ([#104116](https://github.com/kubernetes/kubernetes/pull/104116), [@saschagrunert](https://github.com/saschagrunert)) 2594 - Fixed architecture within manifest for non `amd64` etcd images. ([#105484](https://github.com/kubernetes/kubernetes/pull/105484), [@saschagrunert](https://github.com/saschagrunert)) 2595 - Fixed azure disk translation issue due to lower case `managed` kind. ([#103439](https://github.com/kubernetes/kubernetes/pull/103439), [@andyzhangx](https://github.com/andyzhangx)) 2596 - Fixed client IP preservation for NodePort service with protocol SCTP in ipvs. ([#104756](https://github.com/kubernetes/kubernetes/pull/104756), [@tnqn](https://github.com/tnqn)) 2597 - Fixed concurrent map access causing panics when logging timed-out API calls. ([#105734](https://github.com/kubernetes/kubernetes/pull/105734), [@marseel](https://github.com/marseel)) 2598 - Fixed consolidate logs for `instance not found` error 2599 Fixed skip `not found` nodes when reconciling LB backend address pools ([#105188](https://github.com/kubernetes/kubernetes/pull/105188), [@nilo19](https://github.com/nilo19)) 2600 - Fixed occasional pod cgroup freeze when using cgroup v1 and systemd driver. ([#104528](https://github.com/kubernetes/kubernetes/pull/104528), [@kolyshkin](https://github.com/kolyshkin)) 2601 - Fixed the issue where logging output of kube-scheduler configuration files included line breaks and escape characters. The output also attempted to output the configuration file in one section without showing the user a more readable format. ([#106228](https://github.com/kubernetes/kubernetes/pull/106228), [@sanchayanghosh](https://github.com/sanchayanghosh)) [SIG Scheduling] 2602 - Fixes a bug that could result in the EndpointSlice controller unnecessarily updating EndpointSlices associated with a Service that had Topology Aware Hints enabled. ([#105267](https://github.com/kubernetes/kubernetes/pull/105267), [@llhuii](https://github.com/llhuii)) 2603 - Fixes a regression that could cause panics in LRU caches in controller-manager, kubelet, kube-apiserver, or client-go. ([#104466](https://github.com/kubernetes/kubernetes/pull/104466), [@stbenjam](https://github.com/stbenjam)) 2604 - Fixes an issue where an admission webhook can observe a v1 Pod object that does not have the `defaultMode` field set in the injected service account token volume in kube-api-server. ([#104523](https://github.com/kubernetes/kubernetes/pull/104523), [@liggitt](https://github.com/liggitt)) 2605 - Fixes the `should support building a client with a CSR` E2E test to work with clusters configured with short certificate lifetimes ([#105396](https://github.com/kubernetes/kubernetes/pull/105396), [@liggitt](https://github.com/liggitt)) 2606 - Graceful node shutdown, allow the actual inhibit delay to be greater than the expected inhibit delay. ([#103137](https://github.com/kubernetes/kubernetes/pull/103137), [@wzshiming](https://github.com/wzshiming)) 2607 - Handle Generic Ephemeral Volumes properly in the node limits scheduler filter and the kubelet `hostPath` check. ([#100482](https://github.com/kubernetes/kubernetes/pull/100482), [@pohly](https://github.com/pohly)) 2608 - Headless Services with no selector which were created without dual-stack enabled will be defaulted to RequireDualStack instead of PreferDualStack. This is consistent with such Services which are created with dual-stack enabled. ([#104986](https://github.com/kubernetes/kubernetes/pull/104986), [@thockin](https://github.com/thockin)) 2609 - Ignore `not a vmss instance` error for VMAS nodes in `EnsureBackendPoolDeleted`. ([#105185](https://github.com/kubernetes/kubernetes/pull/105185), [@ialidzhikov](https://github.com/ialidzhikov)) 2610 - Ignore the case when comparing azure tags in service annotation. ([#104705](https://github.com/kubernetes/kubernetes/pull/104705), [@nilo19](https://github.com/nilo19)) 2611 - Ignore the case when updating Azure tags. ([#104593](https://github.com/kubernetes/kubernetes/pull/104593), [@nilo19](https://github.com/nilo19)) 2612 - Introduce a new server run option 'shutdown-send-retry-after'. If true the HTTP Server will continue listening until all non longrunning request(s) in flight have been drained, during this window all incoming requests will be rejected with a status code `429` and a 'Retry-After' response header. ([#101257](https://github.com/kubernetes/kubernetes/pull/101257), [@tkashem](https://github.com/tkashem)) 2613 - Kube-apiserver: Avoid unnecessary repeated calls to `admission webhooks` that reject an update or delete request. ([#104182](https://github.com/kubernetes/kubernetes/pull/104182), [@liggitt](https://github.com/liggitt)) 2614 - Kube-apiserver: Server Side Apply merge order is reverted to match v1.22 behavior until `http://issue.k8s.io/104641` is resolved. ([#106661](https://github.com/kubernetes/kubernetes/pull/106661), [@liggitt](https://github.com/liggitt)) 2615 - Kube-apiserver: events created via the `events.k8s.io` API group for cluster-scoped objects are now permitted in the default namespace as well for compatibility with events clients and the `v1` API ([#100125](https://github.com/kubernetes/kubernetes/pull/100125), [@h4ghhh](https://github.com/h4ghhh)) 2616 - Kube-apiserver: fix a memory leak when deleting multiple objects with a `deletecollection`. ([#105606](https://github.com/kubernetes/kubernetes/pull/105606), [@sxllwx](https://github.com/sxllwx)) 2617 - Kube-proxy health check ports used to listen to `:<port>` for each of the services. This is not needed and opens ports in addresses the cluster user may not have intended. The PR limits listening to all node address which are controlled by `--nodeport-addresses` flag. if no addresses are provided then we default to existing behavior by listening to `:<port>` for each service ([#104742](https://github.com/kubernetes/kubernetes/pull/104742), [@khenidak](https://github.com/khenidak)) 2618 - Kube-proxy: delete stale conntrack UDP entries for loadbalancer ingress IP. ([#104009](https://github.com/kubernetes/kubernetes/pull/104009), [@aojea](https://github.com/aojea)) 2619 - Kube-scheduler now doesn't print any usage message when unknown flag is specified. ([#104503](https://github.com/kubernetes/kubernetes/pull/104503), [@sanposhiho](https://github.com/sanposhiho)) 2620 - Kube-up now includes CoreDNS version v1.8.6 ([#106091](https://github.com/kubernetes/kubernetes/pull/106091), [@rajansandeep](https://github.com/rajansandeep)) [SIG Cloud Provider] 2621 - Kubeadm: When adding an etcd peer to an existing cluster, if an error is returned indicating the peer has already been added, this is accepted and a `ListMembers` call is used instead to return the existing cluster. This helps to diminish the exponential backoff when the first AddMember call times out, while still retaining a similar performance when the peer has already been added from a previous call. ([#104134](https://github.com/kubernetes/kubernetes/pull/104134), [@ihgann](https://github.com/ihgann)) 2622 - Kubeadm: do not allow empty `--config` paths to be passed to `kubeadm kubeconfig user` ([#105649](https://github.com/kubernetes/kubernetes/pull/105649), [@navist2020](https://github.com/navist2020)) 2623 - Kubeadm: fix a bug on Windows worker nodes, where the downloaded KubeletConfiguration from the cluster can contain Linux paths that do not work on Windows and can trip the kubelet binary. ([#105992](https://github.com/kubernetes/kubernetes/pull/105992), [@hwdef](https://github.com/hwdef)) [SIG Cluster Lifecycle and Windows] 2624 - Kubeadm: switch the preflight check (called 'Swap') that verifies if swap is enabled on Linux hosts to report a warning instead of an error. This is related to the graduation of the NodeSwap feature gate in the kubelet to Beta and being enabled by default in 1.23 - allows swap support on Linux hosts. In the next release of kubeadm (1.24) the preflight check will be removed, thus we recommend that you stop using it - e.g. via `--ignore-preflight-errors` or the kubeadm config. ([#104854](https://github.com/kubernetes/kubernetes/pull/104854), [@pacoxu](https://github.com/pacoxu)) 2625 - Kubelet did not report `kubelet_volume_stats_*` metrics for Generic Ephemeral Volumes. ([#105569](https://github.com/kubernetes/kubernetes/pull/105569), [@pohly](https://github.com/pohly)) 2626 - Kubelet's Node Grace Shutdown will terminate probes when shutting down ([#105215](https://github.com/kubernetes/kubernetes/pull/105215), [@rphillips](https://github.com/rphillips)) 2627 - Kubelet: fixes a file descriptor leak in log rotation ([#106382](https://github.com/kubernetes/kubernetes/pull/106382), [@rphillips](https://github.com/rphillips)) [SIG Node] 2628 - Kubelet: the printing of flags at the start of kubelet now uses the final logging configuration. ([#106520](https://github.com/kubernetes/kubernetes/pull/106520), [@pohly](https://github.com/pohly)) 2629 - Make the etcd client (used by the API server) retry certain types of errors. The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72 ([#105069](https://github.com/kubernetes/kubernetes/pull/105069), [@p0lyn0mial](https://github.com/p0lyn0mial)) 2630 - Metrics changes: Fix exposed buckets of `scheduler_volume_scheduling_duration_seconds_bucket` metric. ([#100720](https://github.com/kubernetes/kubernetes/pull/100720), [@dntosas](https://github.com/dntosas)) 2631 - Migrated kubernetes object references (= name + namespace) to structured logging when using JSON as log output format ([#104877](https://github.com/kubernetes/kubernetes/pull/104877), [@pohly](https://github.com/pohly)) 2632 - Pass additional flags to subpath mount to avoid flakes in certain conditions. ([#104253](https://github.com/kubernetes/kubernetes/pull/104253), [@mauriciopoppe](https://github.com/mauriciopoppe)) 2633 - Pod SecurityContext sysctls name parameter for update requests where the existing object's sysctl contains slashes and kubelet sysctl whitelist support contains slashes. ([#102393](https://github.com/kubernetes/kubernetes/pull/102393), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Apps, Auth, Node, Storage and Testing] 2634 - Pod will not start when Init container was OOM killed. ([#104650](https://github.com/kubernetes/kubernetes/pull/104650), [@yxxhero](https://github.com/yxxhero)) [SIG Node] 2635 - PodResources interface was changed, now it returns only isolated CPUs ([#97415](https://github.com/kubernetes/kubernetes/pull/97415), [@AlexeyPerevalov](https://github.com/AlexeyPerevalov)) 2636 - Provide IPv6 support for internal load balancer. ([#103794](https://github.com/kubernetes/kubernetes/pull/103794), [@nilo19](https://github.com/nilo19)) 2637 - Reduce the number of calls to docker for stats via dockershim. For Windows this reduces the latency when calling docker, for Linux this saves cpu cycles. ([#104287](https://github.com/kubernetes/kubernetes/pull/104287), [@jsturtevant](https://github.com/jsturtevant)) [SIG Node and Windows] 2638 - Removed the error message label from the `kubelet_started_pods_errors_total` metric ([#105213](https://github.com/kubernetes/kubernetes/pull/105213), [@yxxhero](https://github.com/yxxhero)) 2639 - Resolves a potential issue with GC and NS controllers which may delete objects after getting a 404 response from the server during its startup. This PR ensures that requests to aggregated APIs will get 503, not 404 while the APIServiceRegistrationController hasn't finished its job. ([#104748](https://github.com/kubernetes/kubernetes/pull/104748), [@p0lyn0mial](https://github.com/p0lyn0mial)) 2640 - Respect grace period when updating static pods. ([#104743](https://github.com/kubernetes/kubernetes/pull/104743), [@gjkim42](https://github.com/gjkim42)) [SIG Node and Testing] 2641 - Revert building binaries with PIE mode. ([#105352](https://github.com/kubernetes/kubernetes/pull/105352), [@ehashman](https://github.com/ehashman)) 2642 - Reverts adding namespace label to admission metrics (and histogram exansion) due to cardinality issues. ([#104033](https://github.com/kubernetes/kubernetes/pull/104033), [@s-urbaniak](https://github.com/s-urbaniak)) 2643 - Reverts the CRI API version surfaced by dockershim to `v1alpha2`. ([#106808](https://github.com/kubernetes/kubernetes/pull/106808), [@saschagrunert](https://github.com/saschagrunert)) 2644 - Scheduler resource metrics over fractional binary quantities (2.5Gi, 1.1Ki) were incorrectly reported as very small values. ([#103751](https://github.com/kubernetes/kubernetes/pull/103751), [@y-tag](https://github.com/y-tag)) 2645 - Support more than 100 disk mounts on Windows ([#105673](https://github.com/kubernetes/kubernetes/pull/105673), [@andyzhangx](https://github.com/andyzhangx)) 2646 - Support using negative array index in JSON patch replace operations. ([#105896](https://github.com/kubernetes/kubernetes/pull/105896), [@zqzten](https://github.com/zqzten)) 2647 - The `--leader-elect*` CLI args are now honored in scheduler. ([#105915](https://github.com/kubernetes/kubernetes/pull/105915), [@Huang-Wei](https://github.com/Huang-Wei)) 2648 - The `--leader-elect*` CLI args are now honored in the scheduler. ([#105712](https://github.com/kubernetes/kubernetes/pull/105712), [@Huang-Wei](https://github.com/Huang-Wei)) 2649 - The `client-go` dynamic client sets the header `Content-Type: application/json` by default ([#104327](https://github.com/kubernetes/kubernetes/pull/104327), [@sxllwx](https://github.com/sxllwx)) 2650 - The `kube-Proxy` now correctly filters out unready endpoints for Services with Topology. ([#106507](https://github.com/kubernetes/kubernetes/pull/106507), [@robscott](https://github.com/robscott)) 2651 - The `pods/binding` subresource now honors `metadata.uid` and `metadata.resourceVersion` ([#105913](https://github.com/kubernetes/kubernetes/pull/105913), [@aholic](https://github.com/aholic)) 2652 - The kube-proxy sync_proxy_rules_iptables_total metric now gives 2653 the correct number of rules, rather than being off by one. 2654 2655 Fixed multiple iptables proxy regressions introduced in 1.22: 2656 2657 - When using Services with SessionAffinity, client affinity for an 2658 endpoint now gets broken when that endpoint becomes non-ready 2659 (rather than continuing until the endpoint is fully deleted). 2660 2661 - Traffic to a service IP now starts getting rejected (as opposed to 2662 merely dropped) as soon as there are no longer any *usable* 2663 endpoints, rather than waiting until all of the terminating 2664 endpoints have terminated even when those terminating endpoints 2665 were not being used. 2666 2667 - Chains for endpoints that won't be used are no longer output to 2668 iptables, saving a bit of memory/time/cpu. ([#106030](https://github.com/kubernetes/kubernetes/pull/106030), [@danwinship](https://github.com/danwinship)) [SIG Network] 2669 - Topology Aware Hints now ignores unready endpoints when assigning hints. ([#106510](https://github.com/kubernetes/kubernetes/pull/106510), [@robscott](https://github.com/robscott)) 2670 - Topology Hints now excludes control plane notes from capacity calculations. ([#104744](https://github.com/kubernetes/kubernetes/pull/104744), [@robscott](https://github.com/robscott)) 2671 - Update Go used to build migrate script in etcd image to v1.16.7. ([#104301](https://github.com/kubernetes/kubernetes/pull/104301), [@serathius](https://github.com/serathius)) 2672 - Updated json representation for a conflicted taint to `Key=Effect` when a conflicted taint occurs in kubectl taint. ([#104011](https://github.com/kubernetes/kubernetes/pull/104011), [@manugupt1](https://github.com/manugupt1)) 2673 - Upgrades functionality of `kubectl kustomize` as described at 2674 https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv4.4.1 ([#106389](https://github.com/kubernetes/kubernetes/pull/106389), [@natasha41575](https://github.com/natasha41575)) [SIG CLI] 2675 - Watch requests that are delegated to aggregated API servers no longer reserve concurrency units (seats) in the API Priority and Fairness dispatcher for their entire duration. ([#105511](https://github.com/kubernetes/kubernetes/pull/105511), [@benluddy](https://github.com/benluddy)) 2676 - When a static pod file is deleted and recreated while using a fixed UID, the pod was not properly restarted. ([#104847](https://github.com/kubernetes/kubernetes/pull/104847), [@smarterclayton](https://github.com/smarterclayton)) 2677 - XFS-filesystems are now force-formatted (option `-f`) in order to avoid problems being formatted due to detection of magic super-blocks. This aligns with the behaviour of formatting of ext3/4 filesystems. ([#104923](https://github.com/kubernetes/kubernetes/pull/104923), [@davidkarlsen](https://github.com/davidkarlsen)) 2678 - `--log-flush-frequency` had no effect in several commands or was missing. Help and warning texts were not always using the right format for a command (`add_dir_header` instead of `add-dir-header`). Fixing this included cleaning up flag handling in component-base/logs: that package no longer adds flags to the global flag sets. Commands which want the klog and `--log-flush-frequency` flags must explicitly call `logs.AddFlags`; the new `cli.Run` does that for commands. That helper function also covers flag normalization and printing of usage and errors in a consistent way (print usage text first if parsing failed, then the error). ([#105076](https://github.com/kubernetes/kubernetes/pull/105076), [@pohly](https://github.com/pohly)) 2679 2680 ### Other (Cleanup or Flake) 2681 2682 - All `klog` flags except for `-v` and `-vmodule` are deprecated. Support for `-vmodule` is only guaranteed for the text log format. ([#105042](https://github.com/kubernetes/kubernetes/pull/105042), [@pohly](https://github.com/pohly)) 2683 - Better pod events ("waiting for ephemeral volume controller to create the persistentvolumeclaim"" instead of "persistentvolumeclaim not found") when using generic ephemeral volumes. ([#104605](https://github.com/kubernetes/kubernetes/pull/104605), [@pohly](https://github.com/pohly)) 2684 - Changed buckets in apiserver_request_duration_seconds metric from [0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0,1.25, 1.5, 1.75, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 40, 50, 60] to [0.05, 0.1, 0.2, 0.4, 0.6, 0.8, 1.0, 1.25, 1.5, 2, 3, 4, 5, 6, 8, 10, 15, 20, 30, 45, 60] ([#106306](https://github.com/kubernetes/kubernetes/pull/106306), [@pawbana](https://github.com/pawbana)) [SIG API Machinery, Instrumentation and Testing] 2685 - Deprecate `apiserver_longrunning_gauge` and `apiserver_register_watchers` in 1.23.0. ([#103793](https://github.com/kubernetes/kubernetes/pull/103793), [@yan-lgtm](https://github.com/yan-lgtm)) 2686 - Enhanced error message for nodes not selected by scheduler due to pod's PersistentVolumeClaim(s) bound to PersistentVolume(s) that do not exist. ([#105196](https://github.com/kubernetes/kubernetes/pull/105196), [@yibozhuang](https://github.com/yibozhuang)) 2687 - Fix an issue in cleaning up `CertificateSigningRequest` objects with an unparseable `status.certificate` field. ([#103823](https://github.com/kubernetes/kubernetes/pull/103823), [@liggitt](https://github.com/liggitt)) 2688 - Kube-apiserver: requests to node, Service, and Pod `/proxy` subresources with no additional URL path now only automatically redirect GET and HEAD requests. ([#95128](https://github.com/kubernetes/kubernetes/pull/95128), [@Riaankl](https://github.com/Riaankl)) 2689 - Kube-apiserver: sets an upper-bound on the lifetime of idle keep-alive connections and the time to read the headers of incoming requests. ([#103958](https://github.com/kubernetes/kubernetes/pull/103958), [@liggitt](https://github.com/liggitt)) 2690 - Kubeadm: external etcd endpoints passed in the `ClusterConfiguration` that have Unicode characters are no longer IDNA encoded (converted to Punycode). They are now just URL encoded as per Go's implementation of RFC-3986, have duplicate "/" removed from the URL paths, and passed like that directly to the `kube-apiserver` `--etcd-servers` flag. If you have etcd endpoints that have Unicode characters, it is advisable to encode them in advance with tooling that is fully IDNA compliant. If you don't do that, the Go standard library (used in k8s and etcd) would do it for you when making requests to the endpoints. ([#103801](https://github.com/kubernetes/kubernetes/pull/103801), [@gkarthiks](https://github.com/gkarthiks)) 2691 - Kubeadm: remove the `--port` flag from the manifest for the `kube-controller-manager` since the flag has been a NO-OP since 1.22 and insecure serving was removed for the component. ([#104157](https://github.com/kubernetes/kubernetes/pull/104157), [@knight42](https://github.com/knight42)) 2692 - Kubeadm: remove the `--port` flag from the manifest for the kube-scheduler since the flag has been a NO-OP since 1.23 and insecure serving was removed for the component. ([#105034](https://github.com/kubernetes/kubernetes/pull/105034), [@pacoxu](https://github.com/pacoxu)) 2693 - Kubeadm: update references to legacy artifacts locations, the `ci-cross` prefix has been removed from the version match as it does not exist in the new `gs://k8s-release-dev` bucket. ([#103813](https://github.com/kubernetes/kubernetes/pull/103813), [@SataQiu](https://github.com/SataQiu)) 2694 - Kubectl: deprecated command line flags (like several of the klog flags) now have a `DEPRECATED: <explanation>` comment. ([#106172](https://github.com/kubernetes/kubernetes/pull/106172), [@pohly](https://github.com/pohly)) [SIG CLI] 2695 - Kubemark is now built as a portable, static binary. ([#106150](https://github.com/kubernetes/kubernetes/pull/106150), [@pohly](https://github.com/pohly)) [SIG Scalability and Testing] 2696 - Migrate `cmd/proxy/{config, healthcheck, winkernel}` to structured logging ([#104944](https://github.com/kubernetes/kubernetes/pull/104944), [@jyz0309](https://github.com/jyz0309)) 2697 - Migrate `pkg/proxy` to structured logging ([#104908](https://github.com/kubernetes/kubernetes/pull/104908), [@CIPHERTron](https://github.com/CIPHERTron)) 2698 - Migrate `pkg/scheduler/framework/plugins/interpodaffinity/filtering.go`,`pkg/scheduler/framework/plugins/podtopologyspread/filtering.go`, `pkg/scheduler/framework/plugins/volumezone/volume_zone.go` to structured logging ([#105931](https://github.com/kubernetes/kubernetes/pull/105931), [@mengjiao-liu](https://github.com/mengjiao-liu)) 2699 - Migrate `pkg/scheduler` to structured logging. ([#99273](https://github.com/kubernetes/kubernetes/pull/99273), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) 2700 - Migrate cmd/proxy/app and pkg/proxy/meta_proxier to structured logging ([#104928](https://github.com/kubernetes/kubernetes/pull/104928), [@jyz0309](https://github.com/jyz0309)) 2701 - Migrated `cmd/kube-scheduler/app/server.go`, `pkg/scheduler/framework/plugins/nodelabel/node_label.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/csi.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/non_csi.go` to structured logging ([#105855](https://github.com/kubernetes/kubernetes/pull/105855), [@shivanshu1333](https://github.com/shivanshu1333)) 2702 - Migrated `pkg/proxy/ipvs` to structured logging ([#104932](https://github.com/kubernetes/kubernetes/pull/104932), [@shivanshu1333](https://github.com/shivanshu1333)) 2703 - Migrated `pkg/proxy/userspace` to structured logging. ([#104931](https://github.com/kubernetes/kubernetes/pull/104931), [@shivanshu1333](https://github.com/shivanshu1333)) 2704 - Migrated `pkg/proxy` to structured logging ([#104891](https://github.com/kubernetes/kubernetes/pull/104891), [@shivanshu1333](https://github.com/shivanshu1333)) 2705 - Migrated `pkg/scheduler/framework/plugins/volumebinding/assume_cache.go` to structured logging. ([#105904](https://github.com/kubernetes/kubernetes/pull/105904), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation, Scheduling and Storage] 2706 - Migrated `pkg/scheduler/framework/preemption/preemption.go`, `pkg/scheduler/framework/plugins/examples/stateful/stateful.go`, and `pkg/scheduler/framework/plugins/noderesources/resource_allocation.go` to structured logging ([#105967](https://github.com/kubernetes/kubernetes/pull/105967), [@shivanshu1333](https://github.com/shivanshu1333)) [SIG Instrumentation, Node and Scheduling] 2707 - Migrated pkg/proxy/winuserspace to structured logging ([#105035](https://github.com/kubernetes/kubernetes/pull/105035), [@shivanshu1333](https://github.com/shivanshu1333)) 2708 - Migrated scheduler file `cache.go` to structured logging ([#105969](https://github.com/kubernetes/kubernetes/pull/105969), [@shivanshu1333](https://github.com/shivanshu1333)) [SIG Instrumentation and Scheduling] 2709 - Migrated scheduler files `comparer.go`, `dumper.go`, `node_tree.go` to structured logging ([#105968](https://github.com/kubernetes/kubernetes/pull/105968), [@shivanshu1333](https://github.com/shivanshu1333)) [SIG Instrumentation and Scheduling] 2710 - More detailed logging has been added to the EndpointSlice controller for Topology. ([#104741](https://github.com/kubernetes/kubernetes/pull/104741), [@robscott](https://github.com/robscott)) 2711 - Remove deprecated and not supported old cronjob controller. ([#106126](https://github.com/kubernetes/kubernetes/pull/106126), [@soltysh](https://github.com/soltysh)) [SIG Apps] 2712 - Remove ignore error flag for drain, and set this feature as default ([#105571](https://github.com/kubernetes/kubernetes/pull/105571), [@yuzhiquan](https://github.com/yuzhiquan)) [SIG CLI] 2713 - Remove the deprecated flags `--csr-only` and `--csr-dir` from `kubeadm certs renew`. Please use `kubeadm certs generate-csr` instead. ([#104796](https://github.com/kubernetes/kubernetes/pull/104796), [@RA489](https://github.com/RA489)) 2714 - Support allocating whole NUMA nodes in the CPUManager when there is not a 1:1 mapping between socket and NUMA node ([#102015](https://github.com/kubernetes/kubernetes/pull/102015), [@klueska](https://github.com/klueska)) 2715 - Support for Windows Server 2022 was added to the `k8s.gcr.io/pause:3.6` image. ([#104711](https://github.com/kubernetes/kubernetes/pull/104711), [@claudiubelu](https://github.com/claudiubelu)) 2716 - Surface warning when users don't set `propagationPolicy` for jobs while deleting. ([#104080](https://github.com/kubernetes/kubernetes/pull/104080), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) 2717 - The `AllowInsecureBackendProxy` feature gate is removed. It reached GA in Kubernetes 1.21. ([#103796](https://github.com/kubernetes/kubernetes/pull/103796), [@mengjiao-liu](https://github.com/mengjiao-liu)) 2718 - The `BoundServiceAccountTokenVolume` feature gate that is GA since v1.22 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. ([#104167](https://github.com/kubernetes/kubernetes/pull/104167), [@ialidzhikov](https://github.com/ialidzhikov)) 2719 - The `StartupProbe` feature gate that is GA since v1.20 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. ([#104168](https://github.com/kubernetes/kubernetes/pull/104168), [@ialidzhikov](https://github.com/ialidzhikov)) 2720 - The `SupportPodPidsLimit` and `SupportNodePidsLimit` feature gates that are GA since v1.20 are unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. ([#104163](https://github.com/kubernetes/kubernetes/pull/104163), [@ialidzhikov](https://github.com/ialidzhikov)) 2721 - The `apiserver` exposes 4 new metrics that allow to track the status of the Service CIDRs allocations: 2722 - current number of available IPs per Service CIDR 2723 - current number of used IPs per Service CIDR 2724 - total number of allocation per Service CIDR 2725 - total number of allocation errors per ServiceCIDR ([#104119](https://github.com/kubernetes/kubernetes/pull/104119), [@aojea](https://github.com/aojea)) 2726 - The flag `--deployment-controller-sync-period` has been deprecated and will be removed in v1.24. ([#103538](https://github.com/kubernetes/kubernetes/pull/103538), [@Pingan2017](https://github.com/Pingan2017)) 2727 - The image `gcr.io/kubernetes-e2e-test-images` will no longer be used in E2E / CI testing, `k8s.gcr.io/e2e-test-images` will be used instead. ([#103724](https://github.com/kubernetes/kubernetes/pull/103724), [@claudiubelu](https://github.com/claudiubelu)) 2728 - The kube-proxy image contains `/go-runner` as a replacement for deprecated klog flags. ([#106301](https://github.com/kubernetes/kubernetes/pull/106301), [@pohly](https://github.com/pohly)) 2729 - The maximum length of the `CSINode` id field has increased to 256 bytes to match the CSI spec. ([#104160](https://github.com/kubernetes/kubernetes/pull/104160), [@pacoxu](https://github.com/pacoxu)) 2730 - Troubleshooting: informers log handlers that take more than 100 milliseconds to process an object if the `DeltaFIFO` queue starts to grow beyond 10 elements. ([#103917](https://github.com/kubernetes/kubernetes/pull/103917), [@aojea](https://github.com/aojea)) 2731 - Update `cri-tools` dependency to v1.22.0. ([#104430](https://github.com/kubernetes/kubernetes/pull/104430), [@saschagrunert](https://github.com/saschagrunert)) 2732 - Update `migratecmd/kube-proxy/app` logs to structured logging. ([#98913](https://github.com/kubernetes/kubernetes/pull/98913), [@yxxhero](https://github.com/yxxhero)) 2733 - Update build images to Debian 11 (Bullseye) 2734 - debian-base:bullseye-v1.0.0 2735 - debian-iptables:bullseye-v1.0.0 2736 - go-runner:v2.3.1-go1.17.1-bullseye.0 2737 - kube-cross:v1.23.0-go1.17.1-bullseye.1 2738 - setcap:bullseye-v1.0.0 2739 - cluster/images/etcd: Build 3.5.0-2 image 2740 - test/conformance/image: Update runner image to base-debian11 ([#105158](https://github.com/kubernetes/kubernetes/pull/105158), [@justaugustus](https://github.com/justaugustus)) 2741 - Update conformance image to use `debian-base:buster-v1.9.0`. ([#104696](https://github.com/kubernetes/kubernetes/pull/104696), [@PushkarJ](https://github.com/PushkarJ)) 2742 - `volume.kubernetes.io/storage-provisioner` annotation will be added to dynamic provisioning required PVC. `volume.beta.kubernetes.io/storage-provisioner` annotation is deprecated. ([#104590](https://github.com/kubernetes/kubernetes/pull/104590), [@Jiawei0227](https://github.com/Jiawei0227)) 2743 2744 ## Dependencies 2745 2746 ### Added 2747 - bazil.org/fuse: 371fbbd 2748 - github.com/OneOfOne/xxhash: [v1.2.2](https://github.com/OneOfOne/xxhash/tree/v1.2.2) 2749 - github.com/antlr/antlr4/runtime/Go/antlr: [b48c857](https://github.com/antlr/antlr4/runtime/Go/antlr/tree/b48c857) 2750 - github.com/cespare/xxhash: [v1.1.0](https://github.com/cespare/xxhash/tree/v1.1.0) 2751 - github.com/cncf/xds/go: [fbca930](https://github.com/cncf/xds/go/tree/fbca930) 2752 - github.com/getkin/kin-openapi: [v0.76.0](https://github.com/getkin/kin-openapi/tree/v0.76.0) 2753 - github.com/go-logr/zapr: [v1.2.0](https://github.com/go-logr/zapr/tree/v1.2.0) 2754 - github.com/google/cel-go: [v0.9.0](https://github.com/google/cel-go/tree/v0.9.0) 2755 - github.com/google/cel-spec: [v0.6.0](https://github.com/google/cel-spec/tree/v0.6.0) 2756 - github.com/google/martian/v3: [v3.1.0](https://github.com/google/martian/v3/tree/v3.1.0) 2757 - github.com/kr/fs: [v0.1.0](https://github.com/kr/fs/tree/v0.1.0) 2758 - github.com/pkg/sftp: [v1.10.1](https://github.com/pkg/sftp/tree/v1.10.1) 2759 - github.com/spaolacci/murmur3: [f09979e](https://github.com/spaolacci/murmur3/tree/f09979e) 2760 - sigs.k8s.io/json: c049b76 2761 2762 ### Changed 2763 - cloud.google.com/go/bigquery: v1.4.0 → v1.8.0 2764 - cloud.google.com/go/storage: v1.6.0 → v1.10.0 2765 - cloud.google.com/go: v0.54.0 → v0.81.0 2766 - github.com/GoogleCloudPlatform/k8s-cloud-provider: [7901bc8 → ea6160c](https://github.com/GoogleCloudPlatform/k8s-cloud-provider/compare/7901bc8...ea6160c) 2767 - github.com/Microsoft/go-winio: [v0.4.15 → v0.4.17](https://github.com/Microsoft/go-winio/compare/v0.4.15...v0.4.17) 2768 - github.com/Microsoft/hcsshim: [5eafd15 → v0.8.22](https://github.com/Microsoft/hcsshim/compare/5eafd15...v0.8.22) 2769 - github.com/benbjohnson/clock: [v1.0.3 → v1.1.0](https://github.com/benbjohnson/clock/compare/v1.0.3...v1.1.0) 2770 - github.com/bketelsen/crypt: [5cbc8cc → v0.0.4](https://github.com/bketelsen/crypt/compare/5cbc8cc...v0.0.4) 2771 - github.com/containerd/cgroups: [0dbf7f0 → v1.0.1](https://github.com/containerd/cgroups/compare/0dbf7f0...v1.0.1) 2772 - github.com/containerd/containerd: [v1.4.4 → v1.4.11](https://github.com/containerd/containerd/compare/v1.4.4...v1.4.11) 2773 - github.com/containerd/continuity: [aaeac12 → v0.1.0](https://github.com/containerd/continuity/compare/aaeac12...v0.1.0) 2774 - github.com/containerd/fifo: [a9fb20d → v1.0.0](https://github.com/containerd/fifo/compare/a9fb20d...v1.0.0) 2775 - github.com/containerd/go-runc: [5a6d9f3 → v1.0.0](https://github.com/containerd/go-runc/compare/5a6d9f3...v1.0.0) 2776 - github.com/containerd/typeurl: [v1.0.1 → v1.0.2](https://github.com/containerd/typeurl/compare/v1.0.1...v1.0.2) 2777 - github.com/coredns/corefile-migration: [v1.0.12 → v1.0.14](https://github.com/coredns/corefile-migration/compare/v1.0.12...v1.0.14) 2778 - github.com/docker/docker: [v20.10.2+incompatible → v20.10.7+incompatible](https://github.com/docker/docker/compare/v20.10.2...v20.10.7) 2779 - github.com/envoyproxy/go-control-plane: [668b12f → 63b5d3c](https://github.com/envoyproxy/go-control-plane/compare/668b12f...63b5d3c) 2780 - github.com/evanphx/json-patch: [v4.11.0+incompatible → v4.12.0+incompatible](https://github.com/evanphx/json-patch/compare/v4.11.0...v4.12.0) 2781 - github.com/go-logr/logr: [v0.4.0 → v1.2.0](https://github.com/go-logr/logr/compare/v0.4.0...v1.2.0) 2782 - github.com/golang/glog: [23def4e → v1.0.0](https://github.com/golang/glog/compare/23def4e...v1.0.0) 2783 - github.com/golang/mock: [v1.4.4 → v1.5.0](https://github.com/golang/mock/compare/v1.4.4...v1.5.0) 2784 - github.com/google/cadvisor: [v0.39.2 → v0.43.0](https://github.com/google/cadvisor/compare/v0.39.2...v0.43.0) 2785 - github.com/google/pprof: [1ebb73c → cbba55b](https://github.com/google/pprof/compare/1ebb73c...cbba55b) 2786 - github.com/hashicorp/golang-lru: [v0.5.1 → v0.5.0](https://github.com/hashicorp/golang-lru/compare/v0.5.1...v0.5.0) 2787 - github.com/ianlancetaylor/demangle: [5e5cf60 → 28f6c0f](https://github.com/ianlancetaylor/demangle/compare/5e5cf60...28f6c0f) 2788 - github.com/json-iterator/go: [v1.1.11 → v1.1.12](https://github.com/json-iterator/go/compare/v1.1.11...v1.1.12) 2789 - github.com/magiconair/properties: [v1.8.1 → v1.8.5](https://github.com/magiconair/properties/compare/v1.8.1...v1.8.5) 2790 - github.com/mitchellh/go-homedir: [v1.1.0 → v1.0.0](https://github.com/mitchellh/go-homedir/compare/v1.1.0...v1.0.0) 2791 - github.com/mitchellh/mapstructure: [v1.1.2 → v1.4.1](https://github.com/mitchellh/mapstructure/compare/v1.1.2...v1.4.1) 2792 - github.com/modern-go/reflect2: [v1.0.1 → v1.0.2](https://github.com/modern-go/reflect2/compare/v1.0.1...v1.0.2) 2793 - github.com/opencontainers/runc: [v1.0.1 → v1.0.2](https://github.com/opencontainers/runc/compare/v1.0.1...v1.0.2) 2794 - github.com/pelletier/go-toml: [v1.2.0 → v1.9.3](https://github.com/pelletier/go-toml/compare/v1.2.0...v1.9.3) 2795 - github.com/prometheus/common: [v0.26.0 → v0.28.0](https://github.com/prometheus/common/compare/v0.26.0...v0.28.0) 2796 - github.com/spf13/afero: [v1.2.2 → v1.6.0](https://github.com/spf13/afero/compare/v1.2.2...v1.6.0) 2797 - github.com/spf13/cast: [v1.3.0 → v1.3.1](https://github.com/spf13/cast/compare/v1.3.0...v1.3.1) 2798 - github.com/spf13/cobra: [v1.1.3 → v1.2.1](https://github.com/spf13/cobra/compare/v1.1.3...v1.2.1) 2799 - github.com/spf13/jwalterweatherman: [v1.0.0 → v1.1.0](https://github.com/spf13/jwalterweatherman/compare/v1.0.0...v1.1.0) 2800 - github.com/spf13/viper: [v1.7.0 → v1.8.1](https://github.com/spf13/viper/compare/v1.7.0...v1.8.1) 2801 - github.com/yuin/goldmark: [v1.3.5 → v1.4.0](https://github.com/yuin/goldmark/compare/v1.3.5...v1.4.0) 2802 - go.opencensus.io: v0.22.3 → v0.23.0 2803 - go.uber.org/zap: v1.17.0 → v1.19.0 2804 - golang.org/x/crypto: 5ea612d → 32db794 2805 - golang.org/x/net: 37e1c6a → e898025 2806 - golang.org/x/oauth2: bf48bf1 → 2bc19b1 2807 - golang.org/x/sys: 59db8d7 → f4d4317 2808 - golang.org/x/term: 6a3ed07 → 6886f2d 2809 - golang.org/x/text: v0.3.6 → v0.3.7 2810 - golang.org/x/tools: v0.1.2 → d4cc65f 2811 - google.golang.org/api: v0.20.0 → v0.46.0 2812 - google.golang.org/appengine: v1.6.5 → v1.6.7 2813 - google.golang.org/genproto: f16073e → fe13028 2814 - google.golang.org/grpc: v1.38.0 → v1.40.0 2815 - google.golang.org/protobuf: v1.26.0 → v1.27.1 2816 - gopkg.in/ini.v1: v1.51.0 → v1.62.0 2817 - honnef.co/go/tools: v0.0.1-2020.1.3 → v0.0.1-2020.1.4 2818 - k8s.io/gengo: b6c5ce2 → 485abfe 2819 - k8s.io/klog/v2: v2.9.0 → v2.30.0 2820 - k8s.io/kube-openapi: 9528897 → e816edb 2821 - k8s.io/system-validators: v1.5.0 → v1.6.0 2822 - k8s.io/utils: 4b05e18 → cb0fa31 2823 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.22 → v0.0.25 2824 - sigs.k8s.io/kustomize/api: v0.8.11 → v0.10.1 2825 - sigs.k8s.io/kustomize/cmd/config: v0.9.13 → v0.10.2 2826 - sigs.k8s.io/kustomize/kustomize/v4: v4.2.0 → v4.4.1 2827 - sigs.k8s.io/kustomize/kyaml: v0.11.0 → v0.13.0 2828 2829 ### Removed 2830 - cloud.google.com/go/datastore: v1.1.0 2831 - cloud.google.com/go/pubsub: v1.2.0 2832 - github.com/alecthomas/units: [f65c72e](https://github.com/alecthomas/units/tree/f65c72e) 2833 - github.com/coreos/bbolt: [v1.3.2](https://github.com/coreos/bbolt/tree/v1.3.2) 2834 - github.com/coreos/etcd: [v3.3.13+incompatible](https://github.com/coreos/etcd/tree/v3.3.13) 2835 - github.com/coreos/go-systemd: [95778df](https://github.com/coreos/go-systemd/tree/95778df) 2836 - github.com/coreos/pkg: [399ea9e](https://github.com/coreos/pkg/tree/399ea9e) 2837 - github.com/dgrijalva/jwt-go: [v3.2.0+incompatible](https://github.com/dgrijalva/jwt-go/tree/v3.2.0) 2838 - github.com/google/martian: [v2.1.0+incompatible](https://github.com/google/martian/tree/v2.1.0) 2839 - github.com/jpillora/backoff: [v1.0.0](https://github.com/jpillora/backoff/tree/v1.0.0) 2840 - gotest.tools: v2.2.0+incompatible 2841 2842 2843 2844 # v1.23.0-rc.1 2845 2846 2847 ## Downloads for v1.23.0-rc.1 2848 2849 ### Source Code 2850 2851 filename | sha512 hash 2852 -------- | ----------- 2853 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes.tar.gz) | 4cf838ebcd3bb756cc604b194dd3b58716b41a34c35f636a7c23af4a501829c7ce23def2fae43547f86bff326bb9268a16b34058e83cada4626930b60a928d97 2854 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-src.tar.gz) | 88af717a64f237de86f287c3715540fcaf6fd9e526a715832b395965ae27187319cc8955434f2511bcace46be455f56d22d569b083bf65f63bd0539fc6ce76a0 2855 2856 ### Client Binaries 2857 2858 filename | sha512 hash 2859 -------- | ----------- 2860 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-darwin-amd64.tar.gz) | 2aa8dbe6fe7926dd78083a243323dd0090f19acb22cc6ab6e7fb74ae3efd494f9927469b466a78eda1a210657935237ae235fbcf99dcc765908e590d8b5979ee 2861 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-darwin-arm64.tar.gz) | 8aaaa11ecdefc349a64b12775656c9f1ed9bfdb72e3dac0f651c629f4f113dc701eb01d07e529c9633bbd5d0eca455a2bf0f654d52830b77917c28ec7bb84dd5 2862 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-linux-386.tar.gz) | 03f482611ec7d12b30c03e382e53b1b00d7263882204d146a0fb74185d19eb90487ee87d898ec8b6c2ed388ac7b31455d79bafc3cf490efacea06825c1ff49c4 2863 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-linux-amd64.tar.gz) | bac04a5d242ff32fbed1a7e756107afd68652ed489c405aee026132f8da57107ea9292ad776ec9f68c5b8a4bc5391a56d6a0ea538461d78778a4f09633fe3bba 2864 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-linux-arm.tar.gz) | cc3ac01dd58f01e8d85b4c0ef70914331addc5b4abde05175d5194d3754c11b7f113a2e2ff79d15a17562565cd7572e643054b67457261563e9f316c3e20a473 2865 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-linux-arm64.tar.gz) | 8fd0109f6ff07656dc0a265248c3fba81cf2faeb3a872dfb4423f14a92bff793e9ca8c330d83a1e957cdf9358287e325dd4f2ea7e52a0bf66ff1f2631b900a2c 2866 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-linux-ppc64le.tar.gz) | 0610276ae835a9b151ad5ac9b192ef7d15b385279c9676ac4f2dfcb0600cb66dd4dcd39e07d65d637f8ae07f0f808f9eddf3312c45c6c435c2db0e0716cf3c29 2867 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-linux-s390x.tar.gz) | bbba68df6895eb722ea7f8678e25e7be6e2d34a50e30c806445515e02b00e3d6982af8e2d54ee757c4fb661fea2f9fa61bd46ffe576897f0583598e3fb0be080 2868 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-windows-386.tar.gz) | 1c41e825e83b0363967af94e759d6f8adb6f77f7f176c804922b089262df933d21528734dd8ebffd3d49fa7196a558a61d178d64f45444708688fe571dd2b4be 2869 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-windows-amd64.tar.gz) | 2b39c66f4d69cdaeff9fa8f46331b2eece1714998df7a9300b6be85e9fec7ccc8cccdc02b3900ab7aa989816242b410234a1b9562b2dd51cff89ae19fd9be491 2870 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-client-windows-arm64.tar.gz) | 77d00160870636703c9604613b6829c3727241616723de638653f5044373ed233608d8680357d16a37ee8fe88f0c158d93da644ddcddd27047f1d76da1a98a68 2871 2872 ### Server Binaries 2873 2874 filename | sha512 hash 2875 -------- | ----------- 2876 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-server-linux-amd64.tar.gz) | d54cbd2ed123ab30ad8662523a9f28af530bba0fe15b22738dbc2a5c3f6917c7ef4e0fcd03eaea36cee38420e968b91607e8d5022598e55030a21777448c57c0 2877 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-server-linux-arm.tar.gz) | 835d94a3833e7b3b257f2174840e974f957d17e026dc2e1781f2d4590738afc947a27982cd06579a711c31dad5933beaa01c3629950dcb3e36f21b4f8e57aad8 2878 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-server-linux-arm64.tar.gz) | 1fed95a3649d87390cb266918f5ed1b8fb00b4ef027cf004f6bffb3d05d1f6f84497c4ac11f34f3d8334d83cab929f46b62122e107ce6c0c6d628c01bb957fc7 2879 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-server-linux-ppc64le.tar.gz) | cb563f6d5625eaa5db4f92b239508c0f30e2c7d9e6200651f23cf59f1962301c2113830e2718c515bcc05cce1a5c1b34d77281c3dc2253e4429e8425a852c238 2880 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-server-linux-s390x.tar.gz) | a382396ddf43b6a4f5e95a95f58c2618cb63e5d5bbe3d8f8c74bdc1f23855eec14e4aa32e6ad6b2d2bfc0ecd2f9da602b2f410f17b0e680dc8f2e69d7f0f0127 2881 2882 ### Node Binaries 2883 2884 filename | sha512 hash 2885 -------- | ----------- 2886 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-node-linux-amd64.tar.gz) | fca9757fe2c67a01841810890fd62fb546c6a7ccb3f475d68a1960a054c0d4db44f71b2669e96d4c7bdb9ac727156448c9eeea64dbdb7442a04e91e5f88fdd76 2887 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-node-linux-arm.tar.gz) | 30a29ba0a71d5cf29a359da78c0c42abc1c478d8cb4736b1e4dfb905770f537e170eba6b1079173867e6973e34b6d51f9d9f86c5573e1e6e7539de0bdd617c7b 2888 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-node-linux-arm64.tar.gz) | 066a437d780b0c871e5635246f9434ba9e7652393546f8b3edba83a56e5a431b4e9aa443a82670d8c5f906bb630426f8a3d8c8b0d146426a20816f588182ea38 2889 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-node-linux-ppc64le.tar.gz) | 8816dc1907a743cf8345acc41f5831d195b8531e313024060a5f322e5ba53061c2878f483e1d7253ed06bdb7edbbcf8784d2f5ef1d6c134e486ee8db9d1852b2 2890 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-node-linux-s390x.tar.gz) | 671e9513d0aeace5352bf22a1522aa64e4f0b041110f6d11663d4fa3c97dc1c60a6197341dd000c8fd35e0cd51b279e748bea04b220dbecd9f589058632e59cc 2891 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.1/kubernetes-node-windows-amd64.tar.gz) | cae457dde9c3bd813b97b2f57912f72cb08668b31700ceb356234ad78183fac45312e9a92e83e094ba7d991152bfe4375d098dfa6db6e3f2675630dd61441039 2892 2893 ## Changelog since v1.23.0-rc.0 2894 2895 ## Changes by Kind 2896 2897 ### Bug or Regression 2898 2899 - Kube-apiserver: Server Side Apply merge order is reverted to match v1.22 behavior until http://issue.k8s.io/104641 is resolved. ([#106661](https://github.com/kubernetes/kubernetes/pull/106661), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] 2900 - Reverts the CRI API version surfaced by dockershim to v1alpha2 ([#106808](https://github.com/kubernetes/kubernetes/pull/106808), [@saschagrunert](https://github.com/saschagrunert)) [SIG Network and Node] 2901 2902 ## Dependencies 2903 2904 ### Added 2905 _Nothing has changed._ 2906 2907 ### Changed 2908 - sigs.k8s.io/structured-merge-diff/v4: v4.2.0 → v4.1.2 2909 2910 ### Removed 2911 _Nothing has changed._ 2912 2913 2914 2915 # v1.23.0-rc.0 2916 2917 2918 ## Downloads for v1.23.0-rc.0 2919 2920 ### Source Code 2921 2922 filename | sha512 hash 2923 -------- | ----------- 2924 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes.tar.gz) | ede62f7d1bde6a11e60b8ff119366c42902090c5b005ca73590856645c16ff12c904cdf45528cb5f48d4ece31db62f8a2c6a2fc4f10d29052c660036f5a47b5b 2925 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-src.tar.gz) | 6103bde6ceeb7b6c40e6e7391731acc4228cf799ee8b7cf612baa8327212a183f16fd560f25b1d608e7f629c230310c585e2e1551436f9569a9d7d5a8c3dbb38 2926 2927 ### Client Binaries 2928 2929 filename | sha512 hash 2930 -------- | ----------- 2931 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-darwin-amd64.tar.gz) | 0266edfb98cf69c62466c87caa1028510cdb0600dfee9f25ba13b6936935011f6b90e8fe6b008a4c8d060012066adff45b28f01cf1c7f9e24293800576c5dc74 2932 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-darwin-arm64.tar.gz) | 8bebf2537a53670a8487ccb43faec62b00439124c99d67ae88c5f1360bb863f03648c430be836580b56dc4526609fd53ef35cabf7a4622c346c87731d5e94575 2933 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-linux-386.tar.gz) | 7cb542707711b5c4cf1402f07d2102b8633b4b75c43be4424f3e15da047badf484b458f6c3e27ecc82343f58f99437c4ab92a0a17f4976f806636d7b070c3396 2934 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-linux-amd64.tar.gz) | eec960213ccb94b1cf1dc47aaf508eb12c1d04c13474b66960db4e0379d23a59aa484bba06cc04e6e7aba22a6e0eb8afd11a35799829dd540767eaedd66b37d1 2935 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-linux-arm.tar.gz) | 7db536134da64c586058603283d752bc0bd7c2ea63b312513fff95d65bc24f978b24c0145446efea6f0b6b8f87f3c74c9a4dd581fa5a186ac630e6d58d30ee9d 2936 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-linux-arm64.tar.gz) | 93a33630fe6bd89fb06f739f7a4184c151c4ac5a8230798b5c3a9137f553f59495e8cc2231f155d6ba51517f70ee095752872c067a12e7bba69ba2b4d9724ee9 2937 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-linux-ppc64le.tar.gz) | afc9ffb6632b4c85837f87d6764e54a8111d4df3a23320294ac0b942dd089789d018faf91a4d5a22ddc4496d9204470267fba09b2a1d6d6f8ad74353df060675 2938 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-linux-s390x.tar.gz) | d5c28f9e65d6a910cf6478342c3e1bd968b16820b2aae6d7ab51d1a646a3b4e46dd7bc2f255f5a02e516d36c90320f4c7b0f836192aae37f45da283fbd76dc88 2939 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-windows-386.tar.gz) | cccb34fd97fb3f05aaf900569bd07772e4ba95f723f7ea71f191926fe05f01b4e608ab49d876b65405051c504043c74b725d09c92321e17a35587703269a37a8 2940 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-windows-amd64.tar.gz) | 61298763df834a36a1f10c0a45cc7c0b520ad38c2f013fd39e0ab11cb07b7f416b9aa695c4289cf21baf9281c3a9ec4e99e93b4b82b368658b91fa19f6486c69 2941 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-client-windows-arm64.tar.gz) | 99d52ce6c6f620464239e4610711daa8d79e0551bab587d9d2342fe315a07fb94b19ffbfdc8f4a769ae51c848f2896517ac3abd256504492bc24389878749775 2942 2943 ### Server Binaries 2944 2945 filename | sha512 hash 2946 -------- | ----------- 2947 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-server-linux-amd64.tar.gz) | fff4bdfce528a16abce7d075570bad7e5fd3b64baa5bd154595b44d25945379e4ea6fb56869681cd2222ad55965214d7e96ea1d32ddd519629a1b943dfbceb2b 2948 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-server-linux-arm.tar.gz) | dc777f74f6d6eef8d56d379cad36e535566993df3abf0be5e00cf22790f01f4336cd757b395aa724b5743db2ad74ce966746755d4970e04a599861f59ea8f12c 2949 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-server-linux-arm64.tar.gz) | a4d935e6816e6bd14e037819949644626813885ef308c7e5ab0a680f71b155cd164c46b963dcdbdcd91cadbf1d0870c66934bb62c9f95698699d1ae3dcc25cb5 2950 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-server-linux-ppc64le.tar.gz) | 9a69761c04556e246e046e18bd4b875e813aef1a1e01931ad0aef61ad11d741d4a1f416949dc9f5a8b7103823f6061557d075a35b4766a8e1192872a3fedb637 2951 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-server-linux-s390x.tar.gz) | 6f64559358d05d659faf79476524101420b6b4e83e27bba11c407624723926adfd14045792a8a874100527d21cf334b56293a94ac1c3e8916b7ec3e9926b52e4 2952 2953 ### Node Binaries 2954 2955 filename | sha512 hash 2956 -------- | ----------- 2957 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-node-linux-amd64.tar.gz) | ed3972e5bb9550d0999b4f4b9da315607bfdb349fae9395e23bb36e72c6cc30aa6df42084b0faa0f5f890b983ed7132357d49d755cde081fe4c53bf78a935f83 2958 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-node-linux-arm.tar.gz) | c87b42358cc75f362eb5a1b52780ee62fa7c18a8760cf7ee744e1910558ffbe5869fea75e49bf8822795c5255ddd935a9364aa067bffeca500b8fb61d70402c5 2959 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-node-linux-arm64.tar.gz) | 34cf8aea703f279559765fdf5c0a079e4679e407134c666082a4ee56d1352c1b66291c847a983ee547a64209143a83a15d89f73d4f2fbb3473dfefb8f9aa630e 2960 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-node-linux-ppc64le.tar.gz) | 86e8bb17b715aea6df3c4ddfd68e4423414ac9a9b86cd38e6272a2226849a456d86496d6154b468b97c22164dc22f80f1dd456a2011ca22f1b782791cf4d3c84 2961 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-node-linux-s390x.tar.gz) | 61723328d454ceade01b1ad5e71bf96caae9badd58b02e877833eafae8465cbc9d09cfd1f414749c4040419bbc594a9256f02a44ce7e762a9753ae50511e57de 2962 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-rc.0/kubernetes-node-windows-amd64.tar.gz) | 3c22248012c2e301209832ded5ecd20d87eb4350a09592763c7de6b46c821b158abbad816c76e012e692ff21ba5da9fb0e861b914166f339f9a0d676c3d4b60a 2963 2964 ## Changelog since v1.23.0-beta.0 2965 2966 ## Changes by Kind 2967 2968 ### API Change 2969 2970 - Add gRPC probe to Pod.Spec.Container.{Liveness,Readiness,Startup}Probe (#106463, @SergeyKanzhelev) [SIG API Machinery, Apps, CLI, Node and Testing] 2971 - Adds a feature gate StatefulSetAutoDeletePVC, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. (#99728, @mattcary) [SIG API Machinery, Apps, Auth and Testing] 2972 - Performs strict server side schema validation requests via the `fieldValidation=[Strict,Warn,Ignore]` query parameter. (#105916, @kevindelgado) [SIG API Machinery, Apps, Auth, Cloud Provider and Testing] 2973 - Support pod priority based node graceful shutdown (#102915, @wzshiming) [SIG Node and Testing] 2974 2975 ### Feature 2976 2977 - CRI v1 is now the project default. If a container runtime does not support the v1 API, Kubernetes will fall back to the v1alpha2 implementation. (#106501, @ehashman) [SIG Network, Node and Testing] 2978 2979 ### Bug or Regression 2980 2981 - Kube-Proxy now correctly filters out unready endpoints for Services with Topology Aware Hints enabled. (#106507, @robscott) [SIG Network] 2982 - Kubelet: the printing of flags at the start of kubelet now uses the final logging configuration (#106520, @pohly) [SIG Node] 2983 - Topology Aware Hints now ignores unready endpoints when assigning hints. (#106510, @robscott) [SIG Apps and Network] 2984 2985 ## Dependencies 2986 2987 ### Added 2988 _Nothing has changed._ 2989 2990 ### Changed 2991 _Nothing has changed._ 2992 2993 ### Removed 2994 _Nothing has changed._ 2995 2996 2997 2998 # v1.23.0-beta.0 2999 3000 3001 ## Downloads for v1.23.0-beta.0 3002 3003 ### Source Code 3004 3005 filename | sha512 hash 3006 -------- | ----------- 3007 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes.tar.gz) | 048cc297840fd70dc571863bbed9da8176a479ca6b8ff17c9a2cc1b1dbf286377d85eb7fccc5d85e1d652658c393ea1eab7ab518631510e1e7462ea638a56b2b 3008 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-src.tar.gz) | 1d3f6f5bb54b61312934169845417dffc428bed0f51342dc2b0eebf7f16899843b0f66f9fb2dcdb2a6e9f25bbdc930ea9adac552b0b011e656151c8cae2f4f71 3009 3010 ### Client Binaries 3011 3012 filename | sha512 hash 3013 -------- | ----------- 3014 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-darwin-amd64.tar.gz) | e22ce7199acf369eacf8422c8ee417041289e927bfc03c238f45faec75c2dabd7f8201c77ed39f20ac311d1ba289766825b7b2f738cfc59b5652a20b98117180 3015 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-darwin-arm64.tar.gz) | 22fa13ca86eb5837db3844b6b7fd134c3ffa3ba5a008635bfa83613a100fa48b3e2331cdf5d368cb267c3cd27e3947fe08ac2540342f1b221192e972695a2cd6 3016 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-linux-386.tar.gz) | 8e239ce934d121b21b534a6d521ca02bf1c6709831e181d103c8d86cdab01b296546be25902162b1060876744f3b579de018b7c2d198e5d5efdd9c849b3ba7ef 3017 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-linux-amd64.tar.gz) | e9355264e3ca91da833fe3c8c1dcc55c287a9b813aad91f26b09e6a75f48be57d12cb235c5f9c6fe2a0aceee09e2b5da84568d81d8002066c8e77d848a03f112 3018 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-linux-arm.tar.gz) | 80e93b6c8cce8221f9a5aba8018fcd95b7ec57728a202fdd158b8df86a733e32d6bb60d8b7ea78da9556058074e9bb88c072b4207a43a4fd2f256cce2593a8df 3019 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-linux-arm64.tar.gz) | 769a1aa41988bbf11a11ef40f42c76740fcbe7fe1fd5d6da948729e1a62bf9c4f28101f47fa9ccd12de50a378b3654e1e4c2d50afad59182c03b8d1e972341e7 3020 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-linux-ppc64le.tar.gz) | 4a9346caef2714f03e65dc3e5e46ade1b311b91ef184b8a47466583e834f44dcdb21c3800793e87c20064b25c3eac2c34637ff6817f1752d52425cdfd5a912fb 3021 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-linux-s390x.tar.gz) | f2129ea05e581a38bdc2771cfdd92ad990620fabf9655f7343c56541a544aa4c6c1e1a2e91a338d06dd0064f35fb5e3027259c317a0909badcbadc9e418c6ced 3022 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-windows-386.tar.gz) | 2dc9459b02f4ed564a7d0e2062e3590c5240debc6a64449d1c714382ded197d5fcf99feecb80ba6483d265ab34126958737cd692783e675b39159be94729c018 3023 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-windows-amd64.tar.gz) | e58cb2f87f619d34afbb2c2c0f2bab484970406216698b79129637cb27c5508b2ca4bd2a3a91847868631bd72947887317692a73fec0f8d67c26aa59868c9d8f 3024 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-client-windows-arm64.tar.gz) | 515bd2e3c95afe613db998ed42ea5456771c488e0963c9fe0328816a6baba09ea4e915d22538e05d478556d17f1678d6a96b75cae25ba742be73da23d04f72ff 3025 3026 ### Server Binaries 3027 3028 filename | sha512 hash 3029 -------- | ----------- 3030 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-server-linux-amd64.tar.gz) | adc6c0e5c07c3e1d24ac4399ea725da5d72a043feaea0063f26188e469b4b8cf537df245015631f1efce9d5e457724858327da3c7c9763f6ca4538aaf77a5e67 3031 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-server-linux-arm.tar.gz) | e6e673cb9baecc56ae03d716569769391cd6f8d38d85810f0199e71b20a4d4c3c92efe7b31a67af463fb01029d94cbcb0c6fe7a0918123055f3fa8f373e76c49 3032 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-server-linux-arm64.tar.gz) | f91dc6e948b702784909ca0c4b8758ad9dbfbcd202ec4e329666b07d42488df00ad64de6a68405668ed881e62e0515271c8168e8316519cd95802239abde4951 3033 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-server-linux-ppc64le.tar.gz) | fbbf3daff8caa89f8249122ba19d67a0d9298fb47d327c0bebd7a54adad4fe6e809164d8bf8e563c79b1f9c8b646f29d18789ec938cbc5746e30649b392c7121 3034 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-server-linux-s390x.tar.gz) | a4ccda542f1b86667e6bf29afd091a2ce6f3a30165ff8b918585fc7794be26d00bd846acaa5b805b270a60df69fbe9827bab6ee472129996e28052bbbe1b0593 3035 3036 ### Node Binaries 3037 3038 filename | sha512 hash 3039 -------- | ----------- 3040 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-node-linux-amd64.tar.gz) | 4d7dd2e50fe65fd1140c51deeb90d8d9f89bbba59502becf626757e2e9eb59fb781bbf3ecb899f1b8e391746329c5c017177287004195387151799e73887f05b 3041 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-node-linux-arm.tar.gz) | d38cd4a06b983a7253d99a6d927c40cbacc636bd73d33172ee03cda502f806638d3cc6f096bc13a55a2faf11ab3e85d77dfd20559e2c880cf54f45ba0875c75c 3042 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-node-linux-arm64.tar.gz) | fa1fa35f30ca589e031485affd2a1016ba5ca0efdf64b35d49c7738342acb55c40733e53fb3b477734bab68d97b00f9adcfb5954ab365169d8f00ac804cc60fb 3043 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-node-linux-ppc64le.tar.gz) | 412b3a133a7711e32455e49d1aac4ce9ee0e44df89afca40dfa8ac52a8aa98649bd4dd7eff85addd8a525bb16b65966dbde1df0c62a994213b4cfa1a7a3b8128 3044 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-node-linux-s390x.tar.gz) | 7e0e217893665a56406b6f1404d616da8578396890b04474fed12ea6b48f5fbf52432efd43c13f66a643284fd54c0fd3441940c777eb1cd0796443fd72d69b6f 3045 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-beta.0/kubernetes-node-windows-amd64.tar.gz) | 768dfe871a028ff7d972d9b59935c1ebdcc8ea0ccf990ee84060ef3bb995ddecb48a49d9fb2ff12dc44ed404d6d9362ee78af3492a4206bb23eb8a0ac8d63ca2 3046 3047 ## Changelog since v1.23.0-alpha.4 3048 3049 ## Urgent Upgrade Notes 3050 3051 ### (No, really, you MUST read this before you upgrade) 3052 3053 - Log messages in JSON format are written to stderr by default now (same as text format) instead of stdout. Users who expected JSON output on stdout must now capture stderr instead or in addition to stdout. (#106146, @pohly) [SIG API Machinery, Architecture, Cluster Lifecycle and Instrumentation] 3054 - [kube-log-runner](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/component-base/logs/kube-log-runner) is included in release tar balls. It can be used to replace the deprecated `--log-file` parameter. (#106123, @pohly) [SIG API Machinery, Architecture, Cloud Provider, Cluster Lifecycle and Instrumentation] 3055 3056 ## Changes by Kind 3057 3058 ### Deprecation 3059 3060 - Kubeadm: add a new output/v1alpha2 API that is identical to the output/v1alpha1, but attempts to resolve some internal dependencies with the kubeadm/v1beta2 API. The output/v1alpha1 API is now deprecated and will be removed in a future release. (#105295, @neolit123) [SIG Cluster Lifecycle] 3061 - Kubeadm: add the kubeadm specific, Alpha (disabled by default) feature gate UnversionedKubeletConfigMap. When this feature is enabled kubeadm will start using a new naming format for the ConfigMap where it stores the KubeletConfiguration structure. The old format included the Kubernetes version - "kube-system/kubelet-config-1.22", while the new format does not - "kube-system/kubelet-config". A similar formatting change is done for the related RBAC rules. The old format is now DEPRECATED and will be removed after the feature graduates to GA. When writing the ConfigMap kubeadm (init, upgrade apply) will respect the value of UnversionedKubeletConfigMap, while when reading it (join, reset, upgrade), it would attempt to use new format first and fallback to the legacy format if needed. (#105741, @neolit123) [SIG Cluster Lifecycle and Testing] 3062 3063 ### API Change 3064 3065 - A new field `omitManagedFields` has been added to both `audit.Policy` and `audit.PolicyRule` 3066 so cluster operators can opt in to omit managed fields of the request and response bodies from 3067 being written to the API audit log. (#94986, @tkashem) [SIG API Machinery, Auth, Cloud Provider and Testing] 3068 - Create HPA v2 from v2beta2 with some fields changed. (#102534, @wangyysde) [SIG API Machinery, Apps, Auth, Autoscaling and Testing] 3069 - Fix kube-proxy regression on UDP services because the logic to detect stale connections was not considering if the endpoint was ready. (#106163, @aojea) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Contributor Experience, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] 3070 - Implement support for recovering from volume expansion failures (#106154, @gnufied) [SIG API Machinery, Apps and Storage] 3071 - In kubelet, log verbosity and flush frequency can also be configured via the configuration file and not just via command line flags. In other commands (kube-apiserver, kube-controller-manager), the flags are listed in the "Logs flags" group and not under "Global" or "Misc". The type for `-vmodule` was made a bit more descriptive (`pattern=N,...` instead of `moduleSpec`). (#106090, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling] 3072 - IngressClass.Spec.Parameters.Namespace field is now GA. (#104636, @hbagdi) [SIG Network and Testing] 3073 - KubeSchedulerConfiguration provides a new field `MultiPoint` which will register a plugin for all valid extension points (#105611, @damemi) [SIG Scheduling and Testing] 3074 - Kubelet should reject pods whose OS doesn't match the node's OS label. (#105292, @ravisantoshgudimetla) [SIG Apps and Node] 3075 - The CSIVolumeFSGroupPolicy feature has moved from beta to GA. (#105940, @dobsonj) [SIG Storage] 3076 - The Kubelet's `--register-with-taints` option is now available via the Kubelet config file field registerWithTaints (#105437, @cmssczy) [SIG Node and Scalability] 3077 - Validation rules for Custom Resource Definitions can be written in the [CEL expression language](https://github.com/google/cel-spec) using the `x-kubernetes-validations` extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (#106051, @jpbetz) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] 3078 3079 ### Feature 3080 3081 - (beta feature) If the CSI driver supports the NodeServiceCapability `VOLUME_MOUNT_GROUP` and the `DelegateFSGroupToCSIDriver` feature gate is enabled, kubelet will delegate applying FSGroup to the driver by passing it to NodeStageVolume and NodePublishVolume, regardless of what other FSGroup policies are set. (#106330, @verult) [SIG Storage] 3082 - /openapi/v3 endpoint will be populated with OpenAPI v3 if the feature flag is enabled (#105945, @Jefftree) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] 3083 - Add support for PodAndContainerStatsFromCRI featuregate, which allows a user to specify their pod stats must also come from the CRI, not cAdvisor. (#103095, @haircommander) [SIG Node] 3084 - Add support for Portworx plugin to csi-translation-lib. Alpha release 3085 3086 Portworx CSI driver is required to enable migration. 3087 This PR adds support of the `CSIMigrationPortworx` feature gate, which can be enabled by: 3088 3089 1. Adding the feature flag to the kube-controller-manager `--feature-gates=CSIMigrationPortworx=true` 3090 2. Adding the feature flag to the kubelet config: 3091 3092 featureGates: 3093 CSIMigrationPortworx: true (#103447, @trierra) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows] 3094 - Added ability for kubectl wait to wait on arbitary JSON path (#105776, @lauchokyip) [SIG CLI] 3095 - Added the ability to specify whether to use an RFC7396 JSON Merge Patch, an RFC6902 JSON Patch, or a Strategic Merge Patch to perform an override of the resources created by kubectl run and kubectl expose. (#105140, @brianpursley) [SIG CLI] 3096 - Adding option for kubectl cp to resume on network errors until completion, requires tar in addition to tail inside the container image (#104792, @matthyx) [SIG CLI] 3097 - Adds --as-uid flag to kubectl to allow uid impersonation in the same way as user and group impersonation. (#105794, @margocrawf) [SIG API Machinery, Auth, CLI and Testing] 3098 - Allows users to prevent garbage collection on pinned images (#103299, @wgahnagl) [SIG Node] 3099 - CSIMigrationGCE feature flag is turned ON by default (#104722, @leiyiz) [SIG Apps, Cloud Provider, Node, Storage and Testing] 3100 - Changed feature CSIMigrationAWS to on by default. This feature requires the AWS EBS CSI driver to be installed. (#106098, @wongma7) [SIG Storage] 3101 - Ensures that volume is deleted from the storage backend when the user tries to delete the PV object manually and the PV ReclaimPolicy is Delete. (#105773, @deepakkinni) [SIG Apps and Storage] 3102 - Graduating `controller_admission_duration_seconds`, `step_admission_duration_seconds`, `webhook_admission_duration_seconds`, `apiserver_current_inflight_requests` and `apiserver_response_sizes` metrics to stable. (#106122, @rezakrimi) [SIG API Machinery, Instrumentation and Testing] 3103 - Graduating `pending_pods`, `preemption_attempts_total`, `preemption_victims` and `schedule_attempts_total` metrics to stable. Also `e2e_scheduling_duration_seconds` is renamed to `scheduling_attempt_duration_seconds` and the latter is graduated to stable. (#105941, @rezakrimi) [SIG Instrumentation, Scheduling and Testing] 3104 - Integration testing now takes periodic Prometheus scrapes from the etcd server. 3105 There is a new script ,`hack/run-prometheus-on-etcd-scrapes.sh`, that runs a containerized Prometheus server against an archive of such scrapes. (#106190, @MikeSpreitzer) [SIG API Machinery and Testing] 3106 - Kube-apiserver: when merging lists, Server Side Apply now prefers the order of the submitted request instead of the existing persisted object (#105983, @jiahuif) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Storage and Testing] 3107 - Kubectl describe namespace now shows Conditions (#106219, @dlipovetsky) [SIG CLI] 3108 - Kubelet should reconcile `kubernetes.io/os` and `kubernetes.io/arch` labels on the node object. The side-effect of this is kubelet would deny admission to pod which has nodeSelector with label `kubernetes.io/os` or `kubernetes.io/arch` which doesn't match the underlying OS or arch on the host OS. 3109 - The label reconciliation happens as part of periodic status update which can be configured via flag `--node-status-update-frequency` (#104613, @ravisantoshgudimetla) [SIG Node, Testing and Windows] 3110 - Kubernetes is now built with Golang 1.17.3 (#106209, @cpanato) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] 3111 - Move ConfigurableFSGroupPolicy to GA 3112 Rename metric volume_fsgroup_recursive_apply to volume_apply_access_control (#105885, @gnufied) [SIG Instrumentation and Storage] 3113 - Moving WindowsHostProcessContainers feature to beta (#106058, @marosset) [SIG Windows] 3114 - The DownwardAPIHugePages feature is now enabled by default. (#106271, @mysunshine92) [SIG Node] 3115 - The PodSecurity admission plugin has graduated to beta and is enabled by default. The admission configuration version has been promoted to pod-security.admission.config.k8s.io/v1beta1. See https://kubernetes.io/docs/concepts/security/pod-security-admission/ for usage guidelines. (#106089, @liggitt) [SIG Auth and Testing] 3116 - This PR adds the following metrics for API Priority and Fairness. 3117 - **apiserver_flowcontrol_priority_level_seat_count_samples**: histograms of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level; the observations are taken once per millisecond. 3118 - **apiserver_flowcontrol_priority_level_seat_count_watermarks**: histograms of high and low watermarks of number of seats occupied by executing requests (both regular and final-delay phases included), broken down by priority_level. 3119 - **apiserver_flowcontrol_watch_count_samples**: histograms of number of watches relevant to a given mutating request, broken down by that request's priority_level and flow_schema. (#105873, @MikeSpreitzer) [SIG API Machinery, Instrumentation and Testing] 3120 - Topology Aware Hints have graduated to beta. (#106433, @robscott) [SIG Network] 3121 - Update the system-validators library to v1.6.0 (#106323, @neolit123) [SIG Cluster Lifecycle and Node] 3122 - Upgrade etcd to 3.5.1 (#105706, @uthark) [SIG Cloud Provider, Cluster Lifecycle and Testing] 3123 - When using `RequestedToCapacityRatio` ScoringStrategy, empty shape will cause error. (#106169, @kerthcet) [SIG Scheduling] 3124 - This release enables in-tree RBD migration to CSI driver with a couple of feature gates. These featuregates are alpha in this release. 3125 3126 - `CSIMigrationRBD`: when enabled, it will redirect traffic from in tree rbd plugin ( kubernetes.io/rbd ) to CSI driver ( rbd.csi.ceph.com) , default to `false` now. 3127 - `IntreePluginRBDUnregister`: Disables the RBD in-tree driver 3128 3129 The feature gates can be enabled by: 3130 3131 1. Adding the feature flag to the kube-controller-manager `--feature-gates=CSIMigrationRBD=true` 3132 2. Adding the feature flag to the kubelet config: 3133 featureGates: 3134 `CSIMigrationRBD`: true 3135 3136 As a Kubernetes cluster operator that administers storage, here are the prerequisites that you must complete before you attempt migration to the RBD CSI driver: 3137 3138 * You must install the Ceph CSI driver (rbd.csi.ceph.com), v3.5.0 or above, into your Kubernetes cluster. 3139 * Considering the clusterID field is a required parameter for CSI driver for its operations, but in-tree StorageClass has monitors field as a required parameter, a Kubernetes storage admin has to create a clusterID based on the monitors hash ( ex:#echo -n '<monitors_string>' | md5sum) in the CSI config map and keep the monitors under this clusterID configuration. 3140 * Also, if the value of adminId in the in-tree Storageclass is different from admin, the adminSecretName mentioned in the in-tree Storageclass has to be patched with the base64 value of the adminId parameter value, otherwise this step can be skipped.(#95361, @humblec) [SIG API Machinery, Node, Scheduling, Storage] 3141 3142 ### Documentation 3143 3144 - Graduating `pod_scheduling_duration_seconds`, `pod_scheduling_attempts`, `framework_extension_point_duration_seconds`, `plugin_execution_duration_seconds` and `queue_incoming_pods_total` metrics to stable. (#106266, @ahg-g) [SIG Instrumentation, Scheduling and Testing] 3145 - Users should not rely on unsupported CRON_TZ variable when specifying schedule, both the API server and cronjob controller will emit warnings pointing to https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/ containing explanation (#106455, @soltysh) [SIG Apps] 3146 3147 ### Bug or Regression 3148 3149 - (PodSecurity admission) errors validating workload resources (deployment, replicaset, etc.) no longer block admission. (#106017, @tallclair) [SIG Auth] 3150 - Add support for Windows Network stats in Containerd (#105744, @jsturtevant) [SIG Node, Testing and Windows] 3151 - Added show-capacity option to `kubectl top node` to show `Capacity` resource usage (#102917, @bysnupy) [SIG CLI] 3152 - Do not unmount and mount subpath bind mounts during container creation unless bind mount changes (#105512, @gnufied) [SIG Storage] 3153 - Don't use a custom dialer for the kubelet if is not rotating certificates, so we can reuse TCP connections and have only one between the apiserver and the kubelet. 3154 If users experiment problems with stale connections using HTTP1.1, they can force the previous behavior of the kubelet by setting the environment variable DISABLE_HTTP2. (#104844, @aojea) [SIG API Machinery, Auth and Node] 3155 - EndpointSlice Mirroring controller now cleans up managed EndpointSlices when a Service selector is added (#105997, @robscott) [SIG Apps, Network and Testing] 3156 - Enhanced event messages for pod failed for exec probe timeout (#106201, @yxxhero) [SIG Node] 3157 - Ensure Pods are removed from the scheduler cache when the scheduler misses deletion events due to transient errors (#106102, @alculquicondor) [SIG Scheduling] 3158 - Fix a panic in kubectl when creating secrets with an improper output type (#106317, @lauchokyip) [SIG CLI] 3159 - Fixed a bug which could cause webhooks to have an incorrect copy of the old object after an Apply or Update (#106195, @alexzielenski) [SIG API Machinery] 3160 - Fixed applying of SELinux labels to CSI volumes on very busy systems (with "error checking for SELinux support: could not get consistent content of /proc/self/mountinfo after 3 attempts") (#105934, @jsafrane) [SIG Storage] 3161 - Fixed bug where using kubectl patch with $deleteFromPrimitiveList on a nonexistent or empty list would add the item to the list (#105421, @brianpursley) [SIG API Machinery] 3162 - Fixed the issue where logging output of kube-scheduler configuration files included line breaks and escape characters. The output also attempted to output the configuration file in one section without showing the user a more readable format. (#106228, @sanchayanghosh) [SIG Scheduling] 3163 - Kube-up now includes CoreDNS version v1.8.6 (#106091, @rajansandeep) [SIG Cloud Provider] 3164 - Kubeadm: fix a bug on Windows worker nodes, where the downloaded KubeletConfiguration from the cluster can contain Linux paths that do not work on Windows and can trip the kubelet binary. (#105992, @hwdef) [SIG Cluster Lifecycle and Windows] 3165 - Kubectl port-forward service will now properly exit when the attached pod dies (#103526, @brianpursley) [SIG API Machinery] 3166 - Kubelet: fixes a file descriptor leak in log rotation (#106382, @rphillips) [SIG Node] 3167 - Pod SecurityContext sysctls name parameter for update requests where the existing object's sysctl contains slashes and kubelet sysctl whitelist support contains slashes. (#102393, @mengjiao-liu) [SIG Apps, Auth, Node, Storage and Testing] 3168 - Pod will not start when Init container was OOM killed. (#104650, @yxxhero) [SIG Node] 3169 - Reduce the number of calls to docker for stats via dockershim. For Windows this reduces the latency when calling docker, for Linux this saves cpu cycles. (#104287, @jsturtevant) [SIG Node and Windows] 3170 - Respect grace period when updating static pods. (#104743, @gjkim42) [SIG Node and Testing] 3171 - The kube-proxy sync_proxy_rules_iptables_total metric now gives 3172 the correct number of rules, rather than being off by one. 3173 3174 Fixed multiple iptables proxy regressions introduced in 1.22: 3175 3176 - When using Services with SessionAffinity, client affinity for an 3177 endpoint now gets broken when that endpoint becomes non-ready 3178 (rather than continuing until the endpoint is fully deleted). 3179 3180 - Traffic to a service IP now starts getting rejected (as opposed to 3181 merely dropped) as soon as there are no longer any *usable* 3182 endpoints, rather than waiting until all of the terminating 3183 endpoints have terminated even when those terminating endpoints 3184 were not being used. 3185 3186 - Chains for endpoints that won't be used are no longer output to 3187 iptables, saving a bit of memory/time/cpu. (#106030, @danwinship) [SIG Network] 3188 - Upgrades functionality of `kubectl kustomize` as described at 3189 https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv4.4.1 (#106389, @natasha41575) [SIG CLI] 3190 3191 ### Other (Cleanup or Flake) 3192 3193 - Changed buckets in apiserver_request_duration_seconds metric from [0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0,1.25, 1.5, 1.75, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 40, 50, 60] to [0.05, 0.1, 0.2, 0.4, 0.6, 0.8, 1.0, 1.25, 1.5, 2, 3, 4, 5, 6, 8, 10, 15, 20, 30, 45, 60] (#106306, @pawbana) [SIG API Machinery, Instrumentation and Testing] 3194 - Kubectl: deprecated command line flags (like several of the klog flags) now have a `DEPRECATED: <explanation>` comment. (#106172, @pohly) [SIG CLI] 3195 - Kubemark is now built as a portable, static binary. (#106150, @pohly) [SIG Scalability and Testing] 3196 - Migrated `pkg/scheduler/framework/plugins/volumebinding/assume_cache.go` to structured logging. (#105904, @mengjiao-liu) [SIG Instrumentation, Scheduling and Storage] 3197 - Migrated `pkg/scheduler/framework/preemption/preemption.go`, `pkg/scheduler/framework/plugins/examples/stateful/stateful.go`, and `pkg/scheduler/framework/plugins/noderesources/resource_allocation.go` to structured logging (#105967, @shivanshu1333) [SIG Instrumentation, Node and Scheduling] 3198 - Migrated scheduler file `cache.go` to structured logging (#105969, @shivanshu1333) [SIG Instrumentation and Scheduling] 3199 - Migrated scheduler files `comparer.go`, `dumper.go`, `node_tree.go` to structured logging (#105968, @shivanshu1333) [SIG Instrumentation and Scheduling] 3200 - Remove deprecated and not supported old cronjob controller. (#106126, @soltysh) [SIG Apps] 3201 - Remove ignore error flag for drain, and set this feature as default (#105571, @yuzhiquan) [SIG CLI] 3202 - The kube-proxy image contains /go-runner as a replacement for deprecated klog flags. (#106301, @pohly) [SIG Testing] 3203 3204 ## Dependencies 3205 3206 ### Added 3207 - github.com/OneOfOne/xxhash: [v1.2.2](https://github.com/OneOfOne/xxhash/tree/v1.2.2) 3208 - github.com/antlr/antlr4/runtime/Go/antlr: [b48c857](https://github.com/antlr/antlr4/runtime/Go/antlr/tree/b48c857) 3209 - github.com/cespare/xxhash: [v1.1.0](https://github.com/cespare/xxhash/tree/v1.1.0) 3210 - github.com/cncf/xds/go: [fbca930](https://github.com/cncf/xds/go/tree/fbca930) 3211 - github.com/getkin/kin-openapi: [v0.76.0](https://github.com/getkin/kin-openapi/tree/v0.76.0) 3212 - github.com/google/cel-go: [v0.9.0](https://github.com/google/cel-go/tree/v0.9.0) 3213 - github.com/google/cel-spec: [v0.6.0](https://github.com/google/cel-spec/tree/v0.6.0) 3214 - github.com/spaolacci/murmur3: [f09979e](https://github.com/spaolacci/murmur3/tree/f09979e) 3215 3216 ### Changed 3217 - github.com/containerd/containerd: [v1.4.9 → v1.4.11](https://github.com/containerd/containerd/compare/v1.4.9...v1.4.11) 3218 - github.com/coredns/corefile-migration: [v1.0.12 → v1.0.14](https://github.com/coredns/corefile-migration/compare/v1.0.12...v1.0.14) 3219 - github.com/docker/docker: [v20.10.2+incompatible → v20.10.7+incompatible](https://github.com/docker/docker/compare/v20.10.2...v20.10.7) 3220 - github.com/envoyproxy/go-control-plane: [668b12f → 63b5d3c](https://github.com/envoyproxy/go-control-plane/compare/668b12f...63b5d3c) 3221 - github.com/golang/glog: [23def4e → v1.0.0](https://github.com/golang/glog/compare/23def4e...v1.0.0) 3222 - github.com/google/cadvisor: [v0.39.2 → v0.43.0](https://github.com/google/cadvisor/compare/v0.39.2...v0.43.0) 3223 - golang.org/x/net: 60bc85c → e898025 3224 - golang.org/x/sys: 41cdb87 → f4d4317 3225 - golang.org/x/text: v0.3.6 → v0.3.7 3226 - google.golang.org/genproto: f16073e → fe13028 3227 - google.golang.org/grpc: v1.38.0 → v1.40.0 3228 - google.golang.org/protobuf: v1.26.0 → v1.27.1 3229 - k8s.io/kube-openapi: 7fbd8d5 → e816edb 3230 - k8s.io/system-validators: v1.5.0 → v1.6.0 3231 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.23 → v0.0.25 3232 - sigs.k8s.io/kustomize/api: v0.8.11 → v0.10.1 3233 - sigs.k8s.io/kustomize/cmd/config: v0.9.13 → v0.10.2 3234 - sigs.k8s.io/kustomize/kustomize/v4: v4.2.0 → v4.4.1 3235 - sigs.k8s.io/kustomize/kyaml: v0.11.0 → v0.13.0 3236 - sigs.k8s.io/structured-merge-diff/v4: v4.1.2 → v4.2.0 3237 3238 ### Removed 3239 _Nothing has changed._ 3240 3241 3242 3243 # v1.23.0-alpha.4 3244 3245 3246 ## Downloads for v1.23.0-alpha.4 3247 3248 ### Source Code 3249 3250 filename | sha512 hash 3251 -------- | ----------- 3252 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes.tar.gz) | aeb10a3fbb89694c52d47203cc958d3543b21426938a9664348163aacd41e20ea7670617a28d8ce6d8d51492980facd5fab062e8ad664dafd7b8dbff1c2bb54f 3253 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-src.tar.gz) | b7a8999335ce15b68360478b22af4daaed10e9db50d597e077d731de194208355d1b2134f5635331d9049dc638d05f1f792d52c5890e521f0af3dc2f3e64fbb8 3254 3255 ### Client Binaries 3256 3257 filename | sha512 hash 3258 -------- | ----------- 3259 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-darwin-amd64.tar.gz) | 5654879ac03f4c7193a8df49cfd4b7253add031c197f50bada40942738bf5720d1c06e31a1d1a7bd1b1a540aa46897e4b34ad8a7e087bd206a7b69b9ffaf5edb 3260 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-darwin-arm64.tar.gz) | 5dce9fee32436c971ef17595f88f3c74f5644ab3af0e3f854a79fb42f3c8d6d8f507fbb0d7b5bcba52ddf1a49ada2559c477278037cf2dadccba72f0398a1093 3261 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-linux-386.tar.gz) | e9eb7dab22801c043da2833fde89d2cd721b9dd622df0ff42b25a6742cfab5cff8bfe3ebbda6cc584cf92db3940b95e25aff935863ed999374ee8923ca0b1215 3262 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-linux-amd64.tar.gz) | 570eeaed029bb05235c58138a777cfd6a4b17d4d91aba346b1fc9a0e573781947599d31a8997e889165db561a18a7ab4d613c2b40a8b2dc0d0225f2411b0fd73 3263 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-linux-arm.tar.gz) | 298923762745cc064a4489aa01d55f57076b84538aef3a6a3554b60257d9959b4eebbb8aeeecdaf14246fa4f1c17750e1b69c63d4940ae71f87010692e41675c 3264 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-linux-arm64.tar.gz) | 498527f1cf2d16af576a6b6d27b5ddbb876e24bd85e34e2c91cf39ef467d366b2059e580fdcccb91e0b61a5f52795273b77ed94a1073b5c0bd574b8661afbe0e 3265 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-linux-ppc64le.tar.gz) | 2632b0fb69565819ef1b6797a834e65f96629df4fd8bec01fce7370672a39afa181854d6ab44afc1c4a6b8143158cf170f5a8e61b75a48071ade2d5ab89d1b2c 3266 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-linux-s390x.tar.gz) | b793a5a8fce9109343ada86f29cf356c6973cd80d81ca47af5c7e4fa11ffccc273f77aba52b1db42ee12abb94ee23677c21910f57c9385646e35742a1c60e17e 3267 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-windows-386.tar.gz) | b92e34ee58e1247c1c444134dd9fa78033d0fda1f51509b43016543596cb211128f8aff730d9a3a9118dfeba139186db2a5dd45455427c7521776e63ee77218c 3268 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-windows-amd64.tar.gz) | 0b5ea6a2de0ff6f71647f428fdbee67c7eb2b918d725cf236ce60daa02e94bd998d15ea0ebb20c4106453e220d11d31506161d5dee3cde6c616dfb5efd11c25e 3269 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-client-windows-arm64.tar.gz) | a4e570be453d1df779bb85c62efb41e98209bb93b57b7655a94a737d552c90f9d3061df9088204c7787344dc6a3eb3f843c58651394c0436d2c90b55e499bfab 3270 3271 ### Server Binaries 3272 3273 filename | sha512 hash 3274 -------- | ----------- 3275 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-server-linux-amd64.tar.gz) | cf215ee7372edd7d5dbf07faee8ccf83de477c8cd431c0fac58357bb8e027349d8edf87364e7db5cec0936f991388f7b183e81e5f92cb6cdf6303efd8cd65d83 3276 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-server-linux-arm.tar.gz) | d6281a6727fcdab956170dca7563fc5099ef79b06c96b2f6bc87fcd0b74f1dea0e14aca344cb41b5b6811919cf4a6d6f60cb08b7fb7034690fd0c4ead82e55ca 3277 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-server-linux-arm64.tar.gz) | 4ec30cdfd8128ca405201c0c40750e10bac016e1e53a7662265328564b09e4feb831a259125bcdd64169d221145cbb166a463216e884dd76f4bb9a72a00e64e0 3278 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-server-linux-ppc64le.tar.gz) | 42b31174a95d0999c78750a1d2c866918c91d11d6406df4e984913f64806708add35c27c0daf255b5d28e98eb815355d1913911f921d34e618dea4d2ebf91949 3279 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-server-linux-s390x.tar.gz) | c4e2b38681c0858d560adc8a330f27e95a035cb0e426c6ff332dcd435cefe88441ea866badab5514c4055191324c48aac108d5d6934a9fd4697da179168b6632 3280 3281 ### Node Binaries 3282 3283 filename | sha512 hash 3284 -------- | ----------- 3285 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-node-linux-amd64.tar.gz) | 708b40c9c0d2cfcb6f9874aa3f1b5a27796cbe2bfe7a2345f381e0d9062df8a6769b2bf29b8b641929ef1f5952897c5739e876e8315eb51cced460b13994c247 3286 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-node-linux-arm.tar.gz) | f89448106af23d6658b9c2e7b43240fb82051d2f89a302ee61fc1cc78e593535993ba12f412c3df907f415e55c38f1783ce9141075198bb9f197b6fa26328d49 3287 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-node-linux-arm64.tar.gz) | bcd8d9fbb244048a3ef3f79f1d4e8f2645bbd69caf353e67ee5c5a4ffd4443da420e5984422933cd4c622c58017a942e20af076f26bfa22f5f38f73a831370ca 3288 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-node-linux-ppc64le.tar.gz) | c0724d053c601d4e80ea19957bd32005aeba0cf8f5e03e8e36412aed0777e860ae680302eef632c8e7d4ef1a8e789e48dc58489ad1a7bf7fd20cb0f755e797af 3289 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-node-linux-s390x.tar.gz) | 0245f592b92d79ccd102961e5b23a9f5b275829e627254fe8ce5f0a7df53ec2c4a9436942686b9d31b696635ab88131cf92e1002869369fa1cf6f080f8073b5f 3290 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.4/kubernetes-node-windows-amd64.tar.gz) | 2a5c6c79ea65f47a42d25b236709a00eafb793e5d87b5f56516da16b85b06e03020679f7cabfb7dc4bc252ff57da0afa52e357915cb1d3801dc3d5c32f096edf 3291 3292 ## Changelog since v1.23.0-alpha.3 3293 3294 ## Changes by Kind 3295 3296 ### Deprecation 3297 3298 - A deprecation notice has been added when using the kube-proxy Userspace proxier, which will be removed in v1.25. (#103860) (#104631, @perithompson) [SIG Network] 3299 - Feature-gate VolumeSubpath has been deprecated and cannot be disabled. It will be completely removed in 1.25 (#105474, @mauriciopoppe) [SIG Storage] 3300 - Kubeadm: remove the deprecated / NO-OP phase "update-cluster-status" in "kubeadm reset" (#105888, @neolit123) [SIG Cluster Lifecycle] 3301 - Removed kubectl --dry-run empty default value and boolean values. kubectl --dry-run usage must be specified with --dry-run=(server|client|none). (#105327, @julianvmodesto) [SIG CLI and Testing] 3302 3303 ### API Change 3304 3305 - #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: 3306 3307 <!-- 3308 This section can be blank if this pull request does not require a release note. 3309 3310 When adding links which point to resources within git repositories, like 3311 KEPs or supporting documentation, please reference a specific commit and avoid 3312 linking directly to the master branch. This ensures that links reference a 3313 specific point in time, rather than a document that may change over time. 3314 3315 See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files 3316 3317 Please use the following format for linking documentation: 3318 - [KEP]: <link> 3319 - [Usage]: <link> 3320 - [Other doc]: <link> 3321 --> (#104782, @kerthcet) [SIG Scheduling and Testing] 3322 - Ephemeral containers have reached beta maturity and are now available by default. (#105405, @verb) [SIG API Machinery, Apps, Node and Testing] 3323 - Introduce OS field in the Pod Spec (#104693, @ravisantoshgudimetla) [SIG API Machinery and Apps] 3324 - Introduce v1beta3 api for scheduler. This version 3325 - increases the weight of user specifiable priorities. 3326 The weights of following priority plugins are increased 3327 - TaintTolerations to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practice 3328 - NodeAffinity to 2 3329 - InterPodAffinity to 2 3330 3331 - Won't have HealthzBindAddress, MetricsBindAddress fields (#104251, @ravisantoshgudimetla) [SIG Scheduling and Testing] 3332 - JSON log output is configurable and now supports writing info messages to stdout and error messages to stderr. Info messages can be buffered in memory. The default is to write both to stdout without buffering, as before. (#104873, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling] 3333 - JobTrackingWithFinalizers graduates to beta. Feature is enabled by default. (#105687, @alculquicondor) [SIG Apps and Testing] 3334 - Remove NodeLease feature gate that was graduated and locked to stable in 1.17 release. (#105222, @cyclinder) [SIG Apps, Node and Testing] 3335 - TTLAfterFinished is now GA and enabled by default (#105219, @sahilvv) [SIG API Machinery, Apps, Auth and Testing] 3336 - The "Generic Ephemeral Volume" feature graduates to GA. It is now enabled unconditionally. (#105609, @pohly) [SIG API Machinery, Apps, Auth, Node, Scheduling, Storage and Testing] 3337 - The legacy scheduler policy config is removed in v1.23, the associated flags policy-config-file, policy-configmap, policy-configmap-namespace and use-legacy-policy-config are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (#105424, @kerthcet) [SIG Scheduling and Testing] 3338 - Track the number of Pods with a Ready condition in Job status. The feature is alpha and needs the feature gate JobReadyPods to be enabled. (#104915, @alculquicondor) [SIG API Machinery, Apps, CLI and Testing] 3339 3340 ### Feature 3341 3342 - Add a new `distribute-cpus-across-numa` option to the static `CPUManager` policy. When enabled, this will trigger the `CPUManager` to evenly distribute CPUs across NUMA nodes in cases where more than one NUMA node is required to satisfy the allocation. (#105631, @klueska) [SIG Node] 3343 - Add support to generate client-side binaries for windows/arm64 platform (#104894, @pacoxu) [SIG CLI, Testing and Windows] 3344 - Added a new feature gate `CustomResourceValidationExpressions` to enable expression validation for Custom Resource. (#105107, @cici37) [SIG API Machinery] 3345 - Adds new [alpha] command 'kubectl events' (#99557, @bboreham) [SIG CLI] 3346 - Client-go, using log level 9, trace the following events of an http request: 3347 - dns lookup 3348 - tcp dialing 3349 - tls handshake 3350 - time to get a connection from the pool 3351 - time to process a request (#105156, @aojea) [SIG API Machinery] 3352 - Client-go: pass `DeleteOptions` down to the fake client `Reactor` (#102945, @chenchun) [SIG API Machinery, Apps and Auth] 3353 - Enhance scheduler volumebinding plugin to handle Lost PVC as UnschedulableAndUnresolvable during PreFilter stage (#105245, @yibozhuang) [SIG Scheduling and Storage] 3354 - Feature-gate `StorageObjectInUseProtection` has been deprecated and cannot be disabled. It will be completely removed in 1.25 (#105495, @ikeeip) [SIG Apps] 3355 - Kubectl will now provide shell completion choices for the --output/-o flag (#105851, @marckhouzam) [SIG CLI] 3356 - Kubernetes is now built with Golang 1.17.2 (#105563, @mengjiao-liu) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] 3357 - Move the getAllocatableResources endpoint in podresource-api to the beta that will make it enabled by default. (#105003, @swatisehgal) [SIG Node and Testing] 3358 - Node affinity, node selector and tolerations are now mutable for jobs that are suspended and have never been started (#105479, @ahg-g) [SIG Apps, Scheduling and Testing] 3359 - Pod template annotations and labels are now mutable for jobs that are suspended and have never been started (#105980, @ahg-g) [SIG Apps] 3360 - PodSecurity: add a container image and manifests for the PodSecurity validating admission webhook (#105923, @liggitt) [SIG Auth] 3361 - PodSecurity: in 1.23+ restricted policy levels, pods and containers which set runAsUser=0 are forbidden at admission-time; previously, they would be rejected at runtime (#105857, @liggitt) [SIG Auth] 3362 - Shell completion now knows to continue suggesting resource names when the command supports it. For example "kubectl get pod pod1 <TAB>" will suggest more pod names. (#105711, @marckhouzam) [SIG CLI] 3363 - Support to enable Hyper-V in GCE Windows Nodes created with kube-up (#105999, @mauriciopoppe) [SIG Cloud Provider and Windows] 3364 - The CPUManager policy options are now enabled, and we introduce a graduation path for the new CPU Manager policy options. (#105012, @fromanirh) [SIG Node and Testing] 3365 - The etcd container image now supports Windows. (#92433, @claudiubelu) [SIG API Machinery and Windows] 3366 - The pods and pod controllers that are exempted from the PodSecurity admission process are now marked with the "pod-security.kubernetes.io/exempt: user/namespace/runtimeClass" annotation, based on what caused the exemption. 3367 3368 The enforcement level that allowed or denied pod during PodSecurity admission is now marked by the "pod-security.kubernetes.io/enforce-policy" annotation. 3369 3370 The annotation that informs about audit policy violations changed from ""pod-security.kubernetes.io/audit" to ""pod-security.kubernetes.io/audit-violation". (#105908, @stlaz) [SIG Auth] 3371 - When feature gate JobTrackingWithFinalizers is enabled: 3372 - Limit the number of pods tracked in a single job sync to avoid starvation of small jobs. 3373 - The metric job_pod_finished_total counts the number of finished pods tracked by the job controller (#105197, @alculquicondor) [SIG Apps, Instrumentation and Testing] 3374 3375 ### Failing Test 3376 3377 - Fixes hostpath storage e2e tests within SELinux enabled env (#104551, @Elbehery) [SIG Testing] 3378 3379 ### Bug or Regression 3380 3381 - (PodSecurity admission) errors validating workload resources (deployment, replicaset, etc.) no longer block admission. (#106017, @tallclair) [SIG Auth] 3382 - Add Pod Security admission metrics: pod_security_evaluations_total, pod_security_exemptions_total, pod_security_errors_total (#105898, @tallclair) [SIG Auth, Instrumentation and Testing] 3383 - Apimachinery: pretty-printed json and yaml output is now indented consistently (#105466, @liggitt) [SIG API Machinery] 3384 - Change `kubectl diff --invalid-arg` status code from 1 to 2 to match docs (#105445, @ardaguclu) [SIG CLI] 3385 - Client-go uses the same http client for all the generated groups and versions, allowing to share customized transports for multiple groups versions. (#105490, @aojea) [SIG API Machinery, Auth, Instrumentation and Testing] 3386 - Evicted and other terminated pods will no longer revert to Running phase (#105462, @ehashman) [SIG Node and Testing] 3387 - Fix pod name of NonIndexed jobs to not include rogue -1 substring (#105676, @alculquicondor) [SIG Apps] 3388 - Fix scoring for NodeResourcesBalancedAllocation plugins when nodes have containers with no requests. (#105845, @ahmad-diaa) [SIG Scheduling] 3389 - Fix: consolidate logs for instance not found error 3390 fix: skip not found nodes when reconciling LB backend address pools (#105188, @nilo19) [SIG Cloud Provider] 3391 - Fix: do not delete the lb that does not exist (#105777, @nilo19) [SIG Cloud Provider] 3392 - Fix: ignore not a VMSS error for VMAS nodes in EnsureBackendPoolDeleted. (#105185, @ialidzhikov) [SIG Cloud Provider] 3393 - Fix: leave the probe path empty for TCP probes (#105253, @nilo19) [SIG Cloud Provider] 3394 - Fix: remove VMSS and VMSS instances from SLB backend pool only when necessary (#105839, @nilo19) [SIG Cloud Provider] 3395 - Fix: skip instance not found when decoupling vmss from lb (#105666, @nilo19) [SIG Cloud Provider] 3396 - Fixed a bug that prevents PersistentVolume that has a Claim UID which doesn't exist in local cache but exists in ETCD from being updated to Released phase. (#105211, @xiaopingrubyist) [SIG Apps] 3397 - Fixed architecture within manifest for non `amd64` etcd images. (#105484, @saschagrunert) [SIG API Machinery] 3398 - Fixes a bug that could result in the EndpointSlice controller unnecessarily updating EndpointSlices associated with a Service that had Topology Aware Hints enabled. (#105267, @llhuii) [SIG Apps and Network] 3399 - Fixes the `should support building a client with a CSR` e2e test to work with clusters configured with short certificate lifetimes (#105396, @liggitt) [SIG Auth and Testing] 3400 - Generic ephemeral volumes can be used also as raw block devices, but the Pod validation was refusing to create pods with that combination. (#105682, @pohly) [SIG Apps, Storage and Testing] 3401 - Generic ephemeral volumes were not considered properly by the node limits scheduler filter and the kubelet hostpath check. (#100482, @pohly) [SIG Node, Scheduling, Storage and Testing] 3402 - Kube-apiserver: fix a memory leak when deleting multiple objects with a deletecollection. (#105606, @sxllwx) [SIG API Machinery] 3403 - Kubeadm: do not allow empty "--config" paths to be passed to "kubeadm kubeconfig user" (#105649, @navist2020) [SIG Cluster Lifecycle] 3404 - Kubelet did not report `kubelet_volume_stats_*` metrics for generic ephemeral voiumes. (#105569, @pohly) [SIG Node] 3405 - Kubelet's Node Grace Shutdown will terminate probes when shutting down. (#105215, @rphillips) [SIG Node] 3406 - Kubernetes object references (= name + namespace) were not logged as struct when using JSON as log output format. (#104877, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] 3407 - Podresources interface was changed, now it returns only isolated cpus (#97415, @AlexeyPerevalov) [SIG Node and Testing] 3408 - Release-note Removed error message label from kubelet_started_pods_errors_total metric (#105213, @yxxhero) [SIG Instrumentation and Node] 3409 - Resolves a potential issue with GC and NS controllers which may delete objects after getting a 404 response from the server during its startup. This PR ensures that requests to aggregated APIs will get 503, not 404 while the APIServiceRegistrationController hasn't finished its job. (#104748, @p0lyn0mial) [SIG API Machinery] 3410 - Revert building binaries with PIE mode. (#105352, @ehashman) [SIG Node, Release and Security] 3411 - Support more than 100 disk mounts on Windows (#105673, @andyzhangx) [SIG Storage and Windows] 3412 - Support using negative array index in json patch replace operations. (#105896, @zqzten) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] 3413 - The --leader-elect* CLI args are now honored in scheduler. (#105712, @Huang-Wei) [SIG Scheduling] 3414 - The client-go dynamic client sets the header 'Content-Type: application/json' by default (#104327, @sxllwx) [SIG API Machinery] 3415 - The pods/binding subresource now honors `metadata.uid` and `metadata.resourceVersion` preconditions (#105913, @aholic) [SIG Scheduling] 3416 - Topology Hints now excludes control plane notes from capacity calculations. (#104744, @robscott) [SIG Apps and Network] 3417 - Watch requests that are delegated to aggregated apiservers no longer reserve concurrency units (seats) in the API Priority and Fairness dispatcher for their entire duration. (#105511, @benluddy) [SIG API Machinery] 3418 - `--log-flush-frequency` had no effect in several commands or was missing. Help and warning texts were not always using the right format for a command (`add_dir_header` instead of `add-dir-header`). Fixing this included cleaning up flag handling in component-base/logs: that package no longer adds flags to the global flag sets. Commands which want the klog and --log-flush-frequency flags must explicitly call logs.AddFlags; the new cli.Run does that for commands. That helper function also covers flag normalization and printing of usage and errors in a consistent way (print usage text first if parsing failed, then the error). (#105076, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling and Testing] 3419 3420 ### Other (Cleanup or Flake) 3421 3422 - All klog flags except for `-v` and `-vmodule` are deprecated. Support for `-vmodule` is only guaranteed for the text log format. (#105042, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle and Instrumentation] 3423 - Kube-apiserver: requests to node, service, and pod `/proxy` subresources with no additional URL path now only automatically redirect GET and HEAD requests. (#95128, @Riaankl) [SIG API Machinery, Architecture and Testing] 3424 - Migrate `pkg/scheduler/framework/plugins/interpodaffinity/filtering.go`,`pkg/scheduler/framework/plugins/podtopologyspread/filtering.go`, `pkg/scheduler/framework/plugins/volumezone/volume_zone.go` to structured logging (#105931, @mengjiao-liu) [SIG Instrumentation and Scheduling] 3425 - Migrated `cmd/kube-scheduler/app/server.go`, `pkg/scheduler/framework/plugins/nodelabel/node_label.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/csi.go`, `pkg/scheduler/framework/plugins/nodevolumelimits/non_csi.go` to structured logging (#105855, @shivanshu1333) [SIG Instrumentation and Scheduling] 3426 - Migrated pkg/proxy to structured logging (#104891, @shivanshu1333) [SIG Network] 3427 - Migrated pkg/proxy/ipvs to structured logging (#104932, @shivanshu1333) [SIG Network] 3428 - Support allocating whole NUMA nodes in the CPUManager when there is not a 1:1 mapping between socket and NUMA node (#102015, @klueska) [SIG Node] 3429 3430 ## Dependencies 3431 3432 ### Added 3433 - sigs.k8s.io/json: c049b76 3434 3435 ### Changed 3436 - github.com/evanphx/json-patch: [v4.11.0+incompatible → v4.12.0+incompatible](https://github.com/evanphx/json-patch/compare/v4.11.0...v4.12.0) 3437 - github.com/go-logr/logr: [v1.1.0 → v1.2.0](https://github.com/go-logr/logr/compare/v1.1.0...v1.2.0) 3438 - github.com/go-logr/zapr: [v1.1.0 → v1.2.0](https://github.com/go-logr/zapr/compare/v1.1.0...v1.2.0) 3439 - k8s.io/klog/v2: v2.20.0 → v2.30.0 3440 - k8s.io/utils: bdf08cb → cb0fa31 3441 3442 ### Removed 3443 _Nothing has changed._ 3444 3445 3446 3447 # v1.23.0-alpha.3 3448 3449 3450 ## Downloads for v1.23.0-alpha.3 3451 3452 ### Source Code 3453 3454 filename | sha512 hash 3455 -------- | ----------- 3456 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes.tar.gz) | 083e6ca03c9d701768b1b5666f354223a3f7dca9fc6410ce45bbf5947152620e300b46df9b6019134e7d736ba44916537eb3bea8fa57e5f7bc3cc34898b4a5dd 3457 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-src.tar.gz) | c3fc74d52e1b7e808c03b9caa30e3e73be30eb8330ce676000b93d5324bbdba93bd005d125b999ba937b79d4751af99b37986911365416f7175d223345f95914 3458 3459 ### Client Binaries 3460 3461 filename | sha512 hash 3462 -------- | ----------- 3463 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-darwin-amd64.tar.gz) | 31d8adc657afbd305df18bfec397a825536357e23b241a19aa538b6ddefefc59743f737db98756e04deea89cc6f260d40a80f02b4d1dc34af1d19e8d796dcd8a 3464 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-darwin-arm64.tar.gz) | b69c4d6cde1c476bafa2ca9916ce3e5bf7286be0ff6a08193bdd1a954ba89b64b1b14193d1acec17ccc141024ee3097971448017b5c9f1327e0961b1e92b2224 3465 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-linux-386.tar.gz) | 059f25ee48aa4b0d1621d6ba87af8fb7e765634d723d98a4e9739f50d3703e7dd3973f4d1ed886c0f3ad6eba165ed81d4e63ecde3b39e66fcbec7d3aa2dfed2e 3466 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-linux-amd64.tar.gz) | 291dba14160803065895799adcde39bdad7a5b0372403f283d6d5e9a094fe1fc79c70e7546f93ee692b9fd297e2667cb558e4209161ecb4bf89965df5746ed4d 3467 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-linux-arm.tar.gz) | 988e12cd7466033578acc487447df376c409e4f79726a4721af1aedbe931e927b22a93d6224891b61b55c7a0ec12e42d8cfcd40e15a9a0cbbc1dbf0e59ab0341 3468 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-linux-arm64.tar.gz) | b3f21dac41b38e671fa7a95892468e2c27fab51abf9c77b336550e5ec213af204e16cac11dd76262fedb0087cf5ad1950af7e36599a38d50cc270cf831cd4f0b 3469 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-linux-ppc64le.tar.gz) | beebf01e2e4ff09bb711284bb9a5c7cc519e4ac8a826dc829394fa28bd9a3149ba73088eaf6712d39a8cab96b0a1c2859e9d5955fee892b759eaddcdeaa8b93c 3470 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-linux-s390x.tar.gz) | 87e5d3d8ba01f9fefb2300e9f06146a254d39d72eaa10cad8c444428b738b3763483ee9eb82f0a13d2ff5aba35fdcb4320598fd5a6a2a07ea3fd00b4ac682d3c 3471 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-windows-386.tar.gz) | 71bfc5a1df9c47735476af10225830212f68c83357ff7d443e18f9b7881524db910781a95d11ff6697cb587352059b5841f7b24fda40b5302ad252bfb6da7e51 3472 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-client-windows-amd64.tar.gz) | 078b0c698f9535f3eee41ecf162d57e2ace67243da36067b78b30cfbb7b27cfcf97af4c5db48cdd592953e26b42b31794002eb96317476849e89e2126c6df99d 3473 3474 ### Server Binaries 3475 3476 filename | sha512 hash 3477 -------- | ----------- 3478 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-server-linux-amd64.tar.gz) | 951b790158dadf46c32e1a1e9c12f2cc8f41e1645602ebff6b4130a08a377bc6d92549186b420332d620d67191123d98a5d717ac0f5ee9643bebe88947ead8fa 3479 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-server-linux-arm.tar.gz) | 0e7a5b9f39b4f45c45bdb5a19dd3695d28f53e1039d76bc572421c707917944d28b1dbfc36e59214b5bc2b93a787900d8e6eb0b587aa801ea8a8faacdb814a4e 3480 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-server-linux-arm64.tar.gz) | 921e060120b8651a0f80977360faca9f207189cee10bc61f669ceba4e540ef48c0ceff1a877ee4c7d31b01b88096bce93c577f68f93b2341c8542dfd89972b60 3481 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-server-linux-ppc64le.tar.gz) | 292cde446b754a87f4ef5384fadbd30017e53ed2744d45a724be467c86ccd9837bfb490db6396642a869937f2f0d080d9655e89ca3345f8365d109a9bcdd18d9 3482 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-server-linux-s390x.tar.gz) | e0ea667f828ce3b36ca4b2a05fb286da5eb321852c50caf0957694553caf2908b27bcc37a5a82277a2606cf6ff4d9e33617ad61628845d9c21f5cf68c960ca92 3483 3484 ### Node Binaries 3485 3486 filename | sha512 hash 3487 -------- | ----------- 3488 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-node-linux-amd64.tar.gz) | e13cd3f75628d354bd1544a5495600fb905741431eb4af4da3d980cc0b7565e3f9c1585d9686cc4e967e54fb854f05bbedfe0c60bb7b855fa027ac8ac45b26e0 3489 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-node-linux-arm.tar.gz) | 6c91b42350528692ff558b667bffd41c5b967c7aa6101471274e4b16b0ac6f84afe01722881328fd4f6f8fe71c7852620fa000186c6f7e56e498fcc2c67ad793 3490 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-node-linux-arm64.tar.gz) | 81728e1388e9cdb436d6847c868f28ab2771331e5e40cd5a7af13cb8dc80a7e4e66a215c12f8183b4884807a3962f913ef5343b889e3c4ecd0e410e8d53aaea9 3491 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-node-linux-ppc64le.tar.gz) | 299649f1b25cc38f3a7543ef4d3ee6d42c85e24ac41b4eb61927bc5c5f0c533a39f9ddd4d5ad1df54c625d77aeb41f6c31b1ca7fd8983262f84fefdf1cb2cfd0 3492 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-node-linux-s390x.tar.gz) | fd6cbc93f98abff9803b43215af6e75a4f7b91ca06969220a779468f34b5ec5ec69f20b529e0cd7b10ba8769bbe2507d46f84ce1d8cd0760380ab9264dd94672 3493 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.3/kubernetes-node-windows-amd64.tar.gz) | a5bfaf2e3ad8d3d2127c3e3e0f131c615a03563253da6bf0e1fd793f6ef71287f341ce1bd0d35eb9a81e0721a5baf03e7c72863b5ed8eb45e8fe70573904ed54 3494 3495 ## Changelog since v1.23.0-alpha.2 3496 3497 ## Changes by Kind 3498 3499 ### Deprecation 3500 3501 - Remove 'master' as a valid EgressSelection type in the EgressSelectorConfiguration API. ([#102242](https://github.com/kubernetes/kubernetes/pull/102242), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery and Cloud Provider] 3502 - Remove VolumeSubpath feature gate ([#105090](https://github.com/kubernetes/kubernetes/pull/105090), [@saad-ali](https://github.com/saad-ali)) [SIG Apps, Node and Storage] 3503 - The deprecated --experimental-bootstrap-kubeconfig flag has been removed. 3504 This can be set via --bootstrap-kubeconfig. ([#103172](https://github.com/kubernetes/kubernetes/pull/103172), [@niulechuan](https://github.com/niulechuan)) [SIG Node] 3505 3506 ### API Change 3507 3508 - Client-go impersonation config can specify a UID to pass impersonated uid information through in requests. ([#104483](https://github.com/kubernetes/kubernetes/pull/104483), [@margocrawf](https://github.com/margocrawf)) [SIG API Machinery, Auth and Testing] 3509 - IPv6DualStack feature moved to stable. 3510 Controller Manager flags for the node IPAM controller have slightly changed: 3511 1. When configuring a dual-stack cluster, the user must specify both --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6 to set the per-node IP mask sizes, instead of the previous --node-cidr-mask-size flag. 3512 2. The --node-cidr-mask-size flag is mutually exclusive with --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6. 3513 3. Single-stack clusters do not need to change, but may choose to use the more specific flags. Users can use either the older --node-cidr-mask-size flag or one of the newer --node-cidr-mask-size-ipv4 or --node-cidr-mask-size-ipv6 flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). ([#104691](https://github.com/kubernetes/kubernetes/pull/104691), [@khenidak](https://github.com/khenidak)) [SIG API Machinery, Apps, Auth, Cloud Provider, Cluster Lifecycle, Network, Node and Testing] 3514 - Kubelet: turn the KubeletConfiguration v1beta1 `ResolverConfig` field from a `string` to `*string`. ([#104624](https://github.com/kubernetes/kubernetes/pull/104624), [@Haleygo](https://github.com/Haleygo)) [SIG Cluster Lifecycle and Node] 3515 3516 ### Feature 3517 3518 - Add mechanism to load simple sniffer class into fluentd-elasticsearch image ([#92853](https://github.com/kubernetes/kubernetes/pull/92853), [@cosmo0920](https://github.com/cosmo0920)) [SIG Cloud Provider and Instrumentation] 3519 - Kubeadm: do not check if the '/etc/kubernetes/manifests' folder is empty on joining worker nodes during preflight ([#104942](https://github.com/kubernetes/kubernetes/pull/104942), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] 3520 - The kube-apiserver's Prometheus metrics have been extended with some that describe the costs of handling LIST requests. They are as follows. 3521 - *apiserver_cache_list_total*: Counter of LIST requests served from watch cache, broken down by resource_prefix and index_name 3522 - *apiserver_cache_list_fetched_objects_total*: Counter of objects read from watch cache in the course of serving a LIST request, broken down by resource_prefix and index_name 3523 - *apiserver_cache_list_evaluated_objects_total*: Counter of objects tested in the course of serving a LIST request from watch cache, broken down by resource_prefix 3524 - *apiserver_cache_list_returned_objects_total*: Counter of objects returned for a LIST request from watch cache, broken down by resource_prefix 3525 - *apiserver_storage_list_total*: Counter of LIST requests served from etcd, broken down by resource 3526 - *apiserver_storage_list_fetched_objects_total*: Counter of objects read from etcd in the course of serving a LIST request, broken down by resource 3527 - *apiserver_storage_list_evaluated_objects_total*: Counter of objects tested in the course of serving a LIST request from etcd, broken down by resource 3528 - *apiserver_storage_list_returned_objects_total*: Counter of objects returned for a LIST request from etcd, broken down by resource ([#104983](https://github.com/kubernetes/kubernetes/pull/104983), [@MikeSpreitzer](https://github.com/MikeSpreitzer)) [SIG API Machinery and Instrumentation] 3529 - Turn on CSIMigrationAzureDisk by default on 1.23 ([#104670](https://github.com/kubernetes/kubernetes/pull/104670), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 3530 3531 ### Bug or Regression 3532 3533 - Changes behaviour of kube-proxy start; does not attempt to set specific sysctl values (which does not work in recent Kernel versions anymore in non-init namespaces), when the current sysctl values are already set higher. ([#103174](https://github.com/kubernetes/kubernetes/pull/103174), [@Napsty](https://github.com/Napsty)) [SIG Network] 3534 - Fix job controller syncs: In case of conflicts, ensure that the sync happens with the most up to date information. Improves reliability of JobTrackingWithFinalizers. ([#105214](https://github.com/kubernetes/kubernetes/pull/105214), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps] 3535 - Fix system default topology spreading when nodes don't have zone labels. Pods correctly spread by default now. ([#105046](https://github.com/kubernetes/kubernetes/pull/105046), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] 3536 - Headless Services with no selector which were created without dual-stack enabled will be defaulted to RequireDualStack instead of PreferDualStack. This is consistent with such Services which are created with dual-stack enabled. ([#104986](https://github.com/kubernetes/kubernetes/pull/104986), [@thockin](https://github.com/thockin)) [SIG Network] 3537 - Kube-apiserver: events created via the `events.k8s.io` API group for cluster-scoped objects are now permitted in the default namespace as well for compatibility with events clients and the `v1` API ([#100125](https://github.com/kubernetes/kubernetes/pull/100125), [@h4ghhh](https://github.com/h4ghhh)) [SIG API Machinery, Apps and Testing] 3538 - Kube-controller incorrectly enabled support for generic ephemeral inline volumes if the storage object in use protection feature was enabled. ([#104913](https://github.com/kubernetes/kubernetes/pull/104913), [@pohly](https://github.com/pohly)) [SIG API Machinery] 3539 - Kubeadm: switch the preflight check (called 'Swap') that verifies if swap is enabled on Linux hosts to report a warning instead of an error. This is related to the graduation of the NodeSwap feature gate in the kubelet to Beta and being enabled by default in 1.23 - allows swap support on Linux hosts. In the next release of kubeadm (1.24) the preflight check will be removed, thus we recommend that you stop using it - e.g. via --ignore-preflight-errors or the kubeadm config. ([#104854](https://github.com/kubernetes/kubernetes/pull/104854), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle] 3540 - Makes the etcd client (used by the API server) retry certain types of errors. The full list of retriable (codes.Unavailable) errors can be found at https://github.com/etcd-io/etcd/blob/main/api/v3rpc/rpctypes/error.go#L72 ([#105069](https://github.com/kubernetes/kubernetes/pull/105069), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery] 3541 - When a static pod file is deleted and recreated while using a fixed UID, the pod was not properly restarted. ([#104847](https://github.com/kubernetes/kubernetes/pull/104847), [@smarterclayton](https://github.com/smarterclayton)) [SIG Node and Testing] 3542 - XFS-filesystems are now force-formatted (option `-f`) in order to avoid problems being formatted due to detection of magic super-blocks. This aligns with the behaviour of formatting of ext3/4 filesystems. ([#104923](https://github.com/kubernetes/kubernetes/pull/104923), [@davidkarlsen](https://github.com/davidkarlsen)) [SIG Storage] 3543 3544 ### Other (Cleanup or Flake) 3545 3546 - Enhanced error message for nodes not selected by scheduler due to pod's PersistentVolumeClaim(s) bound to PersistentVolume(s) that do not exist. ([#105196](https://github.com/kubernetes/kubernetes/pull/105196), [@yibozhuang](https://github.com/yibozhuang)) [SIG Scheduling and Storage] 3547 - Kubeadm: remove the --port flag from the manifest for the kube-scheduler since the flag has been a NO-OP since 1.23 and insecure serving was removed for the component. ([#105034](https://github.com/kubernetes/kubernetes/pull/105034), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle] 3548 - Migrate `cmd/proxy/{config, healthcheck, winkernel}` to structured logging ([#104944](https://github.com/kubernetes/kubernetes/pull/104944), [@jyz0309](https://github.com/jyz0309)) [SIG Network] 3549 - Migrate cmd/proxy/app and pkg/proxy/meta_proxier to structured logging ([#104928](https://github.com/kubernetes/kubernetes/pull/104928), [@jyz0309](https://github.com/jyz0309)) [SIG Apps, Cluster Lifecycle, Network, Node and Testing] 3550 - Migrate pkg/proxy to structured logs ([#104908](https://github.com/kubernetes/kubernetes/pull/104908), [@CIPHERTron](https://github.com/CIPHERTron)) [SIG Network] 3551 - Migrated pkg/proxy/winuserspace to structured logging ([#105035](https://github.com/kubernetes/kubernetes/pull/105035), [@shivanshu1333](https://github.com/shivanshu1333)) [SIG Network] 3552 - The `BoundServiceAccountTokenVolume` feature gate that is GA since v1.22 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. ([#104167](https://github.com/kubernetes/kubernetes/pull/104167), [@ialidzhikov](https://github.com/ialidzhikov)) [SIG Auth] 3553 - The `SupportPodPidsLimit` and `SupportNodePidsLimit` feature gates that are GA since v1.20 are unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. ([#104163](https://github.com/kubernetes/kubernetes/pull/104163), [@ialidzhikov](https://github.com/ialidzhikov)) [SIG Node] 3554 - Update build images to Debian 11 (Bullseye) 3555 - debian-base:bullseye-v1.0.0 3556 - debian-iptables:bullseye-v1.0.0 3557 - go-runner:v2.3.1-go1.17.1-bullseye.0 3558 - kube-cross:v1.23.0-go1.17.1-bullseye.1 3559 - setcap:bullseye-v1.0.0 3560 - cluster/images/etcd: Build 3.5.0-2 image 3561 - test/conformance/image: Update runner image to base-debian11 ([#105158](https://github.com/kubernetes/kubernetes/pull/105158), [@justaugustus](https://github.com/justaugustus)) [SIG API Machinery, Architecture, Release and Testing] 3562 3563 ## Dependencies 3564 3565 ### Added 3566 _Nothing has changed._ 3567 3568 ### Changed 3569 - github.com/json-iterator/go: [v1.1.11 → v1.1.12](https://github.com/json-iterator/go/compare/v1.1.11...v1.1.12) 3570 - github.com/modern-go/reflect2: [v1.0.1 → v1.0.2](https://github.com/modern-go/reflect2/compare/v1.0.1...v1.0.2) 3571 3572 ### Removed 3573 _Nothing has changed._ 3574 3575 3576 3577 # v1.23.0-alpha.2 3578 3579 3580 ## Downloads for v1.23.0-alpha.2 3581 3582 ### Source Code 3583 3584 filename | sha512 hash 3585 -------- | ----------- 3586 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes.tar.gz) | 121d51f42a52b28e27a4b2f914a4f80fa3fba6328e6a4a5c96dec39c5b28c05461fcc290ef35a49058e237091532b24db3cd8c61801bcb6736aee1dd7dbcffc3 3587 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-src.tar.gz) | 641d47241acfadb3b13bccec57795749d2c9e3e07ffa7aa4b30df3a488643631eb8e5cd581bcfb764dff4ac5ed755f72d94e80746142123b09e1675e81421a91 3588 3589 ### Client Binaries 3590 3591 filename | sha512 hash 3592 -------- | ----------- 3593 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-darwin-amd64.tar.gz) | f734cb514ee56adcb2d991a6f0550df907c72f8a61cc2a13117e61b8d5826ff942a582a2e9383deb1a61d5df2243362f1327942a3b4883490eb3296647ce3737 3594 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-darwin-arm64.tar.gz) | 24d1f851cd5782f8f39054e37beda1554dadd8a28cb3272b00d50fc095d1fc3018768c1ea72a44eda61ff0f58f71b33dd28cbdc54467d620e87c3694ecf14cc2 3595 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-linux-386.tar.gz) | 082ad4abea58de3b629fc2ed4560a836cdbeb1adefb0c4cf47044bf33c750d8fcd8a06e2c4ce365853e83a58d52e0129d510a698dd894bd1261f8184dd1cab42 3596 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-linux-amd64.tar.gz) | b3b0b23479c05b57ca574cf17cdcde7e716033bc4f6a80532d1175d8e533e3202bece0dcf503731d5a60319c526ce1ce4a0bc900bf87536321208a59cf890e35 3597 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-linux-arm.tar.gz) | f5dac2976ce04310f74bba6102080554309b851fbd966ff1220d3eb23089db8eb8da519a6bd8865c94f2f24346a4d27eb40fd0a3ff06ca9c6874e1fc6f356b67 3598 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-linux-arm64.tar.gz) | 057b372150749b13a38e04802c7cf566765e0fbb27f1b5f7bf6d3cc3f71eb3020916ea7f8579ecc7fcc10e2db1b5c8caa31a1e8a3aac80da86e4e777f515d42f 3599 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-linux-ppc64le.tar.gz) | 9a090d22aeba011c6d039bff59dbdc23ac4a112828db3cbba588d8b0ee1cd14d16e0eacefbb000e5a3ff26bcce4730824819f86a99b7a9826f35fa9964f9f27a 3600 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-linux-s390x.tar.gz) | 435e20055badb619289dc7c572af300bd2f86068d0b8f326e8d9abfda5347f2449e316158c412e9b946a2541208c3e8cc6e5c823946e74ac4fc2d594d410179a 3601 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-windows-386.tar.gz) | 55f192a4d095d494bb53af1b7133124b762a677eb46247b9dba71d10ea6830b37c30d603908e7a9c63f371baff508b19406e89b231ed5ece0497627f09753f68 3602 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-client-windows-amd64.tar.gz) | 944059d1f1918a793490b95be8130d06189508ba8e79e79ca8cfd2ab98bf396ac551786514b093cc6afe4b3fd15736d728cfcdce18bb32fbee41bc0a97f5c4be 3603 3604 ### Server Binaries 3605 3606 filename | sha512 hash 3607 -------- | ----------- 3608 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-server-linux-amd64.tar.gz) | a76a4b86ee151ba027f7cf4a2072451ae4c829182bb14e00ce1967421744bfc1e58f141b6eaf2ab27ece67054ae307f8e0768477ab9c3c4749eaad397d495182 3609 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-server-linux-arm.tar.gz) | 95aeb4eb473ab4920d81904bc89c6126732b9c6888f9e57493ee99d692042ca44f6844ac1dade1409565f4d9fbec59445402e1f7deac6cbf5b6df16ac814b58c 3610 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-server-linux-arm64.tar.gz) | 3c56e906aafc2a1ac72300352a334662bec5d59e3e523c19b9d65bc52ad9075dc2631f259513efd0f654e220fe0e7d54dfa5028d7eaad81d5d87ca251653f75d 3611 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-server-linux-ppc64le.tar.gz) | b74bacafe9bb6a7cf407747b03e78ae3873e50deec4eaa08758d5e1d5287ac23af59b3ef26f888fe4cd44ccb1455beafcd1384e700230eb445720e3acae5f2e3 3612 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-server-linux-s390x.tar.gz) | d3f8f8d9c233b114129f615252d42782cd366978a49506393a40af3f8b5b1250ce99e9806881675e112a69270a0411fb2f00ea19b99ad7415b9e0074beb2726d 3613 3614 ### Node Binaries 3615 3616 filename | sha512 hash 3617 -------- | ----------- 3618 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-node-linux-amd64.tar.gz) | 146e2f762c179178a57a8c7af7c26470c5d580b8ff8400615162ad1056625f87ce2b32598538d82652f88639e54afb782810529b074c36eb52cc6374414a6181 3619 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-node-linux-arm.tar.gz) | 9357d1b387e1b049fb6cec06a7081afc2ce7e906484c9b061fb0449d147a6c4f9c9dc7a9219cdca5ed71df6c73784f360018d9e48d4fa2aa7eeabef60649d7a4 3620 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-node-linux-arm64.tar.gz) | 8394f8f9d6ee823cb9a470ea67e15d4d0c6aca7065fe826788f50955905373fc3cdddd6db43901c07736588d8d6a3d3e2916bc8d45fd6bd06307583686137a0a 3621 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-node-linux-ppc64le.tar.gz) | 7211cb426834484bff39f1ab3c9541203429039f8f5e522ca9e28c43da749e197128a3cae28db0467fc339305d2f23f85e8b4ed9ec116506c3d8076744a88d5e 3622 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-node-linux-s390x.tar.gz) | a7c1a38250398171d3df5865749e9928867c4f44106ae66d44cf9f948ce4f4eed9d1f273a5d369996425b1e12482fceccde4c7652770a8c9fb3f161811323b69 3623 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.2/kubernetes-node-windows-amd64.tar.gz) | 2007b3b16597cc06b486f87f35b6c637404f07c11d88b8c8e1c2c9bbea97f762bd7d4f9a31f42f78a917c595af5cb89e6885dd88f3766836dc6e4ec79cf084f2 3624 3625 ## Changelog since v1.23.0-alpha.1 3626 3627 ## Changes by Kind 3628 3629 ### Deprecation 3630 3631 - Controller-manager: the following flags have no effect and would be removed in v1.24: 3632 - `--port` 3633 - `--address` 3634 The insecure port flags `--port` may only be set to 0 now. 3635 Also `metricsBindAddress` and `healthzBindAddress` fields from `kubescheduler.config.k8s.io/v1beta1` are no-op and expected to be empty. Removed in `kubescheduler.config.k8s.io/v1beta2` completely. 3636 3637 In addition, please be careful that: 3638 - kube-scheduler MUST start with `--authorization-kubeconfig` and `--authentication-kubeconfig` correctly set to get authentication/authorization working. 3639 - liveness/readiness probes to kube-scheduler MUST use HTTPS now, and the default port has been changed to 10259. 3640 - Applications that fetch metrics from kube-scheduler should use a dedicated service account which is allowed to access nonResourceURLs `/metrics`. ([#96345](https://github.com/kubernetes/kubernetes/pull/96345), [@ingvagabund](https://github.com/ingvagabund)) [SIG Cloud Provider, Scheduling and Testing] 3641 - Removed deprecated metric `scheduler_volume_scheduling_duration_seconds` ([#104518](https://github.com/kubernetes/kubernetes/pull/104518), [@dntosas](https://github.com/dntosas)) [SIG Instrumentation, Scheduling and Storage] 3642 3643 ### API Change 3644 3645 - A small regression in Service updates was fixed. The circumstances are so unlikely that probably nobody would ever hit it. ([#104601](https://github.com/kubernetes/kubernetes/pull/104601), [@thockin](https://github.com/thockin)) [SIG Network] 3646 - Introduce v1beta2 for Priority and Fairness with no changes in API spec ([#104399](https://github.com/kubernetes/kubernetes/pull/104399), [@tkashem](https://github.com/tkashem)) [SIG API Machinery and Testing] 3647 - Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums. ([#104969](https://github.com/kubernetes/kubernetes/pull/104969), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Apps and Network] 3648 - Kubelet: turn the KubeletConfiguration v1beta1 `ResolverConfig` field from a `string` to `*string`. ([#104624](https://github.com/kubernetes/kubernetes/pull/104624), [@Haleygo](https://github.com/Haleygo)) [SIG Cluster Lifecycle and Node] 3649 - Kubernetes is now built using go1.17 ([#103692](https://github.com/kubernetes/kubernetes/pull/103692), [@justaugustus](https://github.com/justaugustus)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing] 3650 - Removed deprecated `--seccomp-profile-root`/`seccompProfileRoot` config ([#103941](https://github.com/kubernetes/kubernetes/pull/103941), [@saschagrunert](https://github.com/saschagrunert)) [SIG Node] 3651 - Since golang 1.17 both net.ParseIP and net.ParseCIDR rejects leading zeros in the dot-decimal notation of IPv4 addresses. 3652 Kubernetes will keep allowing leading zeros on IPv4 address to not break the compatibility. 3653 IMPORTANT: Kubernetes interprets leading zeros on IPv4 addresses as decimal, users must not rely on parser alignment to not being impacted by the associated security advisory: 3654 CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.16.2 and below standard library "net" results in indeterminate SSRF & RFI vulnerabilities. 3655 Reference: https://nvd.nist.gov/vuln/detail/CVE-2021-29923 ([#104368](https://github.com/kubernetes/kubernetes/pull/104368), [@aojea](https://github.com/aojea)) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage and Testing] 3656 - StatefulSet minReadySeconds is promoted to beta ([#104045](https://github.com/kubernetes/kubernetes/pull/104045), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Apps and Testing] 3657 - The `Service.spec.ipFamilyPolicy` field is now *required* in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either `ipFamilies` or `clusterIPs`, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify `ipFamilyPolicy` as either "PreferDualStack" or "RequireDualStack". ([#96684](https://github.com/kubernetes/kubernetes/pull/96684), [@thockin](https://github.com/thockin)) [SIG API Machinery, Apps, Network and Testing] 3658 - Users of LogFormatRegistry in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no `v` field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). ([#104103](https://github.com/kubernetes/kubernetes/pull/104103), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] 3659 - When creating an object with generateName, if a conflict occurs the server now returns an AlreadyExists error with a retry option. ([#104699](https://github.com/kubernetes/kubernetes/pull/104699), [@vincepri](https://github.com/vincepri)) [SIG API Machinery] 3660 3661 ### Feature 3662 3663 - Add fish shell completion to kubectl ([#92989](https://github.com/kubernetes/kubernetes/pull/92989), [@WLun001](https://github.com/WLun001)) [SIG CLI] 3664 - Added PowerShell completion generation by running `kubectl completion powershell` ([#103758](https://github.com/kubernetes/kubernetes/pull/103758), [@zikhan](https://github.com/zikhan)) [SIG CLI] 3665 - Added a `Processing` condition for the workqueue API 3666 Changed `Shutdown` for the workqueue API to wait until the work queue finishes processing all in-flight items. ([#101928](https://github.com/kubernetes/kubernetes/pull/101928), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG API Machinery and Apps] 3667 - Added a new flag `--append-server-path` to `kubectl proxy` that will automatically append the kube context server path to each request. ([#97350](https://github.com/kubernetes/kubernetes/pull/97350), [@FabianKramm](https://github.com/FabianKramm)) [SIG API Machinery, CLI and Testing] 3668 - Added support for setting controller-manager log level online ([#104571](https://github.com/kubernetes/kubernetes/pull/104571), [@h4ghhh](https://github.com/h4ghhh)) [SIG API Machinery, Apps and Cloud Provider] 3669 - Adding support for multiple --from-env-file flags ([#104232](https://github.com/kubernetes/kubernetes/pull/104232), [@lauchokyip](https://github.com/lauchokyip)) [SIG CLI] 3670 - Cloud providers can set service account names for cloud controllers. ([#103178](https://github.com/kubernetes/kubernetes/pull/103178), [@nckturner](https://github.com/nckturner)) [SIG API Machinery and Cloud Provider] 3671 - Health check of kube-controller-manager now includes each controller. ([#104667](https://github.com/kubernetes/kubernetes/pull/104667), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery and Cloud Provider] 3672 - Kubernetes is now built with Golang 1.17.1 ([#104904](https://github.com/kubernetes/kubernetes/pull/104904), [@cpanato](https://github.com/cpanato)) [SIG API Machinery, Cloud Provider, Instrumentation, Release and Testing] 3673 - The pause image list now contains Windows Server 2022 ([#104438](https://github.com/kubernetes/kubernetes/pull/104438), [@nick5616](https://github.com/nick5616)) [SIG Windows] 3674 - Updates debian-iptables to v1.6.7 to pick up CVE fixes ([#104970](https://github.com/kubernetes/kubernetes/pull/104970), [@PushkarJ](https://github.com/PushkarJ)) [SIG API Machinery, Network, Release, Security and Testing] 3675 3676 ### Documentation 3677 3678 - Conformance: the test "[sig-network] EndpointSlice should have Endpoints and EndpointSlices pointing to API Server [Conformance]" only requires that there is an EndpointSlice that references the "kubernetes.default" service, it no longer requires that its named "kubernetes". ([#104664](https://github.com/kubernetes/kubernetes/pull/104664), [@aojea](https://github.com/aojea)) [SIG Architecture, Network and Testing] 3679 3680 ### Bug or Regression 3681 3682 - A pod that the Kubelet rejects was still considered as being accepted for a brief period of time after rejection, which might cause some pods to be rejected briefly that could fit on the node. A pod that is still terminating (but has status indicating it has failed) may also still be consuming resources and so should also be considered. ([#104817](https://github.com/kubernetes/kubernetes/pull/104817), [@smarterclayton](https://github.com/smarterclayton)) [SIG Node] 3683 - Changed kubectl describe to compute Age of an event using the count and lastObservedTime fields available in the event series ([#104482](https://github.com/kubernetes/kubernetes/pull/104482), [@harjas27](https://github.com/harjas27)) [SIG CLI] 3684 - Don't prematurely close reflectors in case of slow initialization in watch based manager to fix issues with inability to properly mount secrets/configmaps. ([#104604](https://github.com/kubernetes/kubernetes/pull/104604), [@wojtek-t](https://github.com/wojtek-t)) [SIG Node] 3685 - Fix Job tracking with finalizers for more than 500 pods, ensuring all finalizers are removed before counting the Pod. ([#104666](https://github.com/kubernetes/kubernetes/pull/104666), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps and Instrumentation] 3686 - Fix a regression where the Kubelet failed to exclude already completed pods from calculations about how many resources it was currently using when deciding whether to allow more pods. ([#104577](https://github.com/kubernetes/kubernetes/pull/104577), [@smarterclayton](https://github.com/smarterclayton)) [SIG Node] 3687 - Fix detach disk issue on deleting vmss node ([#104572](https://github.com/kubernetes/kubernetes/pull/104572), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider] 3688 - Fix: ensure InstanceShutdownByProviderID return false for creating Azure VMs ([#104382](https://github.com/kubernetes/kubernetes/pull/104382), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 3689 - Fix: ignore the case when comparing azure tags in service annotation ([#104705](https://github.com/kubernetes/kubernetes/pull/104705), [@nilo19](https://github.com/nilo19)) [SIG Cloud Provider] 3690 - Fix: ignore the case when updating Azure tags ([#104593](https://github.com/kubernetes/kubernetes/pull/104593), [@nilo19](https://github.com/nilo19)) [SIG Cloud Provider] 3691 - Fixed bug where kubectl would emit duplicate warning messages for flag names that contain an underscore and recommend using a nonexistent flag in some cases ([#103852](https://github.com/kubernetes/kubernetes/pull/103852), [@brianpursley](https://github.com/brianpursley)) [SIG CLI and Cluster Lifecycle] 3692 - Fixed client IP preservation for NodePort service with protocol SCTP in ipvs mode ([#104756](https://github.com/kubernetes/kubernetes/pull/104756), [@tnqn](https://github.com/tnqn)) [SIG Network] 3693 - Fixed occasional pod cgroup freeze when using cgroup v1 and systemd driver. ([#104528](https://github.com/kubernetes/kubernetes/pull/104528), [@kolyshkin](https://github.com/kolyshkin)) [SIG Node] 3694 - Fixes a regression that could cause panics in LRU caches in controller-manager, kubelet, kube-apiserver, or client-go ([#104466](https://github.com/kubernetes/kubernetes/pull/104466), [@stbenjam](https://github.com/stbenjam)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage] 3695 - Kube-apiserver: fixes an issue where an admission webhook can observe a v1 Pod object that does not have the `defaultMode` field set in the injected service account token volume ([#104523](https://github.com/kubernetes/kubernetes/pull/104523), [@liggitt](https://github.com/liggitt)) [SIG Auth] 3696 - Kube-proxy health check ports used to listen to :<port> for each of the services. This is not needed and opens ports in addresses the cluster user may not have intended. The PR limits listening to all node address which are controlled by `--nodeport-addresses` flag. if no addresses are provided then we default to existing behavior by listening to :<port> for each service ([#104742](https://github.com/kubernetes/kubernetes/pull/104742), [@khenidak](https://github.com/khenidak)) [SIG Network] 3697 - Kube-scheduler now doesn't print any usage message when unknown flag is specified ([#104503](https://github.com/kubernetes/kubernetes/pull/104503), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling] 3698 - Metrics changes: Fix exposed buckets of `scheduler_volume_scheduling_duration_seconds_bucket` metric ([#100720](https://github.com/kubernetes/kubernetes/pull/100720), [@dntosas](https://github.com/dntosas)) [SIG Apps, Instrumentation, Scheduling and Storage] 3699 - Scheduler resource metrics over fractional binary quantities (2.5Gi, 1.1Ki) were incorrectly reported as very small values. ([#103751](https://github.com/kubernetes/kubernetes/pull/103751), [@y-tag](https://github.com/y-tag)) [SIG API Machinery and Scheduling] 3700 3701 ### Other (Cleanup or Flake) 3702 3703 - Generic ephemeral volumes: better pod events ("waiting for ephemeral volume controller to create the persistentvolumeclaim"" instead of "persistentvolumeclaim not found") ([#104605](https://github.com/kubernetes/kubernetes/pull/104605), [@pohly](https://github.com/pohly)) [SIG Scheduling and Storage] 3704 - Kubeadm: remove the deprecated flags "--csr-only" and "--csr-dir" from "kubeadm certs renew". Please use "kubeadm certs generate-csr" instead. ([#104796](https://github.com/kubernetes/kubernetes/pull/104796), [@RA489](https://github.com/RA489)) [SIG Cluster Lifecycle] 3705 - Migrate `pkg/scheduler` to structured logging ([#99273](https://github.com/kubernetes/kubernetes/pull/99273), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) [SIG Scheduling] 3706 - Migrated pkg/proxy/userspace to structured logging ([#104931](https://github.com/kubernetes/kubernetes/pull/104931), [@shivanshu1333](https://github.com/shivanshu1333)) [SIG Network] 3707 - More detailed logging has been added to the EndpointSlice controller for Topology Aware Hints. ([#104741](https://github.com/kubernetes/kubernetes/pull/104741), [@robscott](https://github.com/robscott)) [SIG Apps and Network] 3708 - Support for Windows Server 2022 was added to the k8s.gcr.io/pause:3.6 image. ([#104711](https://github.com/kubernetes/kubernetes/pull/104711), [@claudiubelu](https://github.com/claudiubelu)) [SIG CLI, Cloud Provider, Cluster Lifecycle, Node, Release and Testing] 3709 - The maximum length of the CSINode id field has increased to 256 bytes to match the CSI spec ([#104160](https://github.com/kubernetes/kubernetes/pull/104160), [@pacoxu](https://github.com/pacoxu)) [SIG Storage] 3710 - Update conformance image to use debian-base:buster-v1.9.0 ([#104696](https://github.com/kubernetes/kubernetes/pull/104696), [@PushkarJ](https://github.com/PushkarJ)) [SIG Architecture, Release, Security and Testing] 3711 - `volume.kubernetes.io/storage-provisioner` annotation will be added to dynamic provisioning required PVC. `volume.beta.kubernetes.io/storage-provisioner` annotation is deprecated. ([#104590](https://github.com/kubernetes/kubernetes/pull/104590), [@Jiawei0227](https://github.com/Jiawei0227)) [SIG Apps and Storage] 3712 3713 ## Dependencies 3714 3715 ### Added 3716 - bazil.org/fuse: 371fbbd 3717 - github.com/go-logr/zapr: [v1.1.0](https://github.com/go-logr/zapr/tree/v1.1.0) 3718 - github.com/kr/fs: [v0.1.0](https://github.com/kr/fs/tree/v0.1.0) 3719 - github.com/pkg/sftp: [v1.10.1](https://github.com/pkg/sftp/tree/v1.10.1) 3720 3721 ### Changed 3722 - github.com/Microsoft/go-winio: [v0.4.15 → v0.4.17](https://github.com/Microsoft/go-winio/compare/v0.4.15...v0.4.17) 3723 - github.com/Microsoft/hcsshim: [5eafd15 → v0.8.22](https://github.com/Microsoft/hcsshim/compare/5eafd15...v0.8.22) 3724 - github.com/benbjohnson/clock: [v1.0.3 → v1.1.0](https://github.com/benbjohnson/clock/compare/v1.0.3...v1.1.0) 3725 - github.com/bketelsen/crypt: [5cbc8cc → v0.0.4](https://github.com/bketelsen/crypt/compare/5cbc8cc...v0.0.4) 3726 - github.com/containerd/cgroups: [0dbf7f0 → v1.0.1](https://github.com/containerd/cgroups/compare/0dbf7f0...v1.0.1) 3727 - github.com/containerd/containerd: [v1.4.4 → v1.4.9](https://github.com/containerd/containerd/compare/v1.4.4...v1.4.9) 3728 - github.com/containerd/continuity: [aaeac12 → v0.1.0](https://github.com/containerd/continuity/compare/aaeac12...v0.1.0) 3729 - github.com/containerd/fifo: [a9fb20d → v1.0.0](https://github.com/containerd/fifo/compare/a9fb20d...v1.0.0) 3730 - github.com/containerd/go-runc: [5a6d9f3 → v1.0.0](https://github.com/containerd/go-runc/compare/5a6d9f3...v1.0.0) 3731 - github.com/containerd/typeurl: [v1.0.1 → v1.0.2](https://github.com/containerd/typeurl/compare/v1.0.1...v1.0.2) 3732 - github.com/go-logr/logr: [v0.4.0 → v1.1.0](https://github.com/go-logr/logr/compare/v0.4.0...v1.1.0) 3733 - github.com/magiconair/properties: [v1.8.1 → v1.8.5](https://github.com/magiconair/properties/compare/v1.8.1...v1.8.5) 3734 - github.com/mitchellh/go-homedir: [v1.1.0 → v1.0.0](https://github.com/mitchellh/go-homedir/compare/v1.1.0...v1.0.0) 3735 - github.com/mitchellh/mapstructure: [v1.1.2 → v1.4.1](https://github.com/mitchellh/mapstructure/compare/v1.1.2...v1.4.1) 3736 - github.com/opencontainers/runc: [v1.0.1 → v1.0.2](https://github.com/opencontainers/runc/compare/v1.0.1...v1.0.2) 3737 - github.com/pelletier/go-toml: [v1.2.0 → v1.9.3](https://github.com/pelletier/go-toml/compare/v1.2.0...v1.9.3) 3738 - github.com/spf13/afero: [v1.2.2 → v1.6.0](https://github.com/spf13/afero/compare/v1.2.2...v1.6.0) 3739 - github.com/spf13/cast: [v1.3.0 → v1.3.1](https://github.com/spf13/cast/compare/v1.3.0...v1.3.1) 3740 - github.com/spf13/cobra: [v1.1.3 → v1.2.1](https://github.com/spf13/cobra/compare/v1.1.3...v1.2.1) 3741 - github.com/spf13/jwalterweatherman: [v1.0.0 → v1.1.0](https://github.com/spf13/jwalterweatherman/compare/v1.0.0...v1.1.0) 3742 - github.com/spf13/viper: [v1.7.0 → v1.8.1](https://github.com/spf13/viper/compare/v1.7.0...v1.8.1) 3743 - github.com/yuin/goldmark: [v1.3.5 → v1.4.0](https://github.com/yuin/goldmark/compare/v1.3.5...v1.4.0) 3744 - go.uber.org/zap: v1.17.0 → v1.19.0 3745 - golang.org/x/crypto: 5ea612d → 32db794 3746 - golang.org/x/net: abc4532 → 60bc85c 3747 - golang.org/x/oauth2: f6687ab → 2bc19b1 3748 - golang.org/x/sys: 59db8d7 → 41cdb87 3749 - golang.org/x/term: 6a3ed07 → 6886f2d 3750 - golang.org/x/tools: v0.1.2 → d4cc65f 3751 - gopkg.in/ini.v1: v1.51.0 → v1.62.0 3752 - k8s.io/klog/v2: v2.9.0 → v2.20.0 3753 - k8s.io/utils: efc7438 → bdf08cb 3754 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.22 → v0.0.23 3755 3756 ### Removed 3757 - github.com/coreos/bbolt: [v1.3.2](https://github.com/coreos/bbolt/tree/v1.3.2) 3758 - github.com/coreos/etcd: [v3.3.13+incompatible](https://github.com/coreos/etcd/tree/v3.3.13) 3759 - github.com/coreos/go-systemd: [95778df](https://github.com/coreos/go-systemd/tree/95778df) 3760 - github.com/coreos/pkg: [399ea9e](https://github.com/coreos/pkg/tree/399ea9e) 3761 - github.com/dgrijalva/jwt-go: [v3.2.0+incompatible](https://github.com/dgrijalva/jwt-go/tree/v3.2.0) 3762 - gotest.tools: v2.2.0+incompatible 3763 3764 3765 3766 # v1.23.0-alpha.1 3767 3768 3769 ## Downloads for v1.23.0-alpha.1 3770 3771 ### Source Code 3772 3773 filename | sha512 hash 3774 -------- | ----------- 3775 [kubernetes.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes.tar.gz) | f7c76f1e077b5d98019347b2c9b79eaa0c79d428542b9c15dab23886c276ca16314f200ca37af914c52264c0e1e5d0bde639d6adf37368d5e7b29d230df00d95 3776 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-src.tar.gz) | f267f26eca20cd7018e68abeeed38aed5c10dbbae7c531c4e08e507196a4dd3f511eb8d41ee8b09495544337d8e1940a8ca04e94084f8dd172698a96564fb070 3777 3778 ### Client Binaries 3779 3780 filename | sha512 hash 3781 -------- | ----------- 3782 [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-darwin-amd64.tar.gz) | deb110839c2c3cf94ca9b29df2f0b07b3fad6937d7bb6e9d2516d01345c8e324f6ab86fe1d34f1443f04c3d1fc328b53b3d756c295f4ed22f1994071fbc8c9cb 3783 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-darwin-arm64.tar.gz) | 1473cb9fc4847b0daff6c9e3189ce55fadc22fb6190161e744e5438066a714cb467fdebfb35f6445a27f5010df94ee602fff492a2382e0f308fda111d53af1f4 3784 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-linux-386.tar.gz) | ed5f5b0777ca51790d185764afc2c812f82ae27c35d897570fc86cabee90dc0a445d9d8c37c981bd3684ba9cd47dc0d75d0094578e79ef7b591d3c1b6564280f 3785 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-linux-amd64.tar.gz) | 39f2a888e7a43c9e4a4018301894786f6babe23d79ab7a143e06444f69bc14aec2e158d355c5b48da4356e7bd72ec9b1268f8b12815c8b709395f36ad9a68a2f 3786 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-linux-arm.tar.gz) | b6b8333d8adb4bc6a943bcd2c6cd1a0aeaf0b926d06aa03b759e3c723c81ccc91804debc64fedcd7d678eefdee9bdacc52b2891bd084a15fd5f7918a70e51a15 3787 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-linux-arm64.tar.gz) | 3cb8217b9a5363cebad4989253e02c8a37259b61eafc2f08681508c11c5f68448cad43282257c3d90ad510cc9a62645b7f1adeb99fedf5e13c181495e3754ee4 3788 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-linux-ppc64le.tar.gz) | e411700fb13b25deca6347983cdafe47199f0df00086ccd7b3e7d52a7b3bee7e96a85c2568dd52c956fd4ea8b4a6991859c57c9b73a13e06440b456c65b11687 3789 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-linux-s390x.tar.gz) | 6c1395792a175de77436352d0893476363497b0f6a616f4415f91aed5e780d1f25b515021939a7563046237c7b651caba0d1fbf7c4c461677d1b9308b227e94c 3790 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-windows-386.tar.gz) | f3aec7136c21d24a99145ce294a859078fcbf11bae132b8b4081555a6656c0d95ccbaca02a86dc257d557ecebc0673d0771b9cdd10593712a643e8cc0f61d681 3791 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-client-windows-amd64.tar.gz) | b29697ba0a25f3d871ffbe5800dcb23ec9fd27c0122a284e17c21f1258f7dd9d341813aeb7826159c7999581a16db19fbb6eeeab48f5c89975df7595d19102c3 3792 3793 ### Server Binaries 3794 3795 filename | sha512 hash 3796 -------- | ----------- 3797 [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-server-linux-amd64.tar.gz) | a5b3edca559b84cd9d22b43b23d0607951d434e185dcb313b831604d83dd306cfc017599994d3944ce77360116024eb59a302851325bb2c29c185a80db2e6eac 3798 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-server-linux-arm.tar.gz) | 2334dbcff3ba22a50f252998eb63991b6c816659dbaa5f749370fc1b1f78f0af7739e50ab64c14a23c4e7dfa8917568e2a3b85bdffdb2cc691ee23ae8f5c8326 3799 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-server-linux-arm64.tar.gz) | 58674443ce6e359a995dd7c4289bf730e616bcaf336837b77333a206d4e98693d9356a0a670ffbe0b274e2997a8b76a164153cf084f0ff5f91f40f00b5512684 3800 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-server-linux-ppc64le.tar.gz) | f60ebdd04e2348b1ba51540cad93fa24cb133fd25db97150000bffaff8ccb41e1b6506bcde6b7d913aee7701478f975a97775430a82980105383fdb1cc13d260 3801 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-server-linux-s390x.tar.gz) | ff008aa0ba1bf755f32c7251c6aceb12b6f9de00d2e2729302b51960e70e486bd82da62d21d70ad81c14e01910ab2afe0fd2509ebfdec050d36f88ee1f0330b2 3802 3803 ### Node Binaries 3804 3805 filename | sha512 hash 3806 -------- | ----------- 3807 [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-node-linux-amd64.tar.gz) | 352502f10fbc4579bd9556e3f73ca7513184371ea563d12a39d655d39bb14ccf0f485f4f2b54a77d984c91ff0de2acea7225f98532a1247da5b9ecc65081bc1a 3808 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-node-linux-arm.tar.gz) | af9de95e2b9e4c1f39cb9757d4dca020f7d276b6702302a2d92e7a93e9986528615ce54531e62b96f6e8a0b9863cddbb264f42b1f59374948ac3499af60d9532 3809 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-node-linux-arm64.tar.gz) | 45a286cb1d469b16d046af02047cf63a8407222e4a39fe696f5652e0587e0c9ffbdbab6505ce85e2726ba10db3189a7fbe70e316bc610caedc8cbb49fed28076 3810 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-node-linux-ppc64le.tar.gz) | 7a540a3ff0295998a1679b0ccd50cb1825faf1d0afd6ed08138ab3767c83a2743aa43b122c8da89ee00161f57c0af8d76012e890f9fe6d77b4ee8aff4e32e50f 3811 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-node-linux-s390x.tar.gz) | 3cd7656221ac2fa161abcf237878cff26c1d97cf77d9b784736c97a56841397ff859e43947d81a83f8fe4164701da41a1dad69b551c4e1fee49b3f8196878236 3812 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.23.0-alpha.1/kubernetes-node-windows-amd64.tar.gz) | 21e63913024e88a48244a598cd400fbae6ce8f8910202f1b635812fbc9281b7c6097eb10a321dd18846484a198845bba58970d83b5119a367862cf8418d4d08c 3813 3814 ## Changelog since v1.22.0 3815 3816 ## Urgent Upgrade Notes 3817 3818 ### (No, really, you MUST read this before you upgrade) 3819 3820 - #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: 3821 3822 <!-- 3823 This section can be blank if this pull request does not require a release note. 3824 3825 When adding links which point to resources within git repositories, like 3826 KEPs or supporting documentation, please reference a specific commit and avoid 3827 linking directly to the master branch. This ensures that links reference a 3828 specific point in time, rather than a document that may change over time. 3829 3830 See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files 3831 3832 Please use the following format for linking documentation: 3833 - [KEP]: <link> 3834 - [Usage]: <link> 3835 - [Other doc]: <link> 3836 --> ([#104389](https://github.com/kubernetes/kubernetes/pull/104389), [@saschagrunert](https://github.com/saschagrunert)) [SIG Node] 3837 - Kubeadm: remove the deprecated flag --experimental-patches for the init|join|upgrade commands. The flag --patches is no longer allowed in a mixture with the flag --config. Please use the kubeadm configuration for setting patches for a node using {Init|Join}Configuration.patches. ([#104065](https://github.com/kubernetes/kubernetes/pull/104065), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle] 3838 3839 ## Changes by Kind 3840 3841 ### Deprecation 3842 3843 - Add apiserver_longrunning_requests metric to replace the soon to be deprecated apiserver_longrunning_gauge metric. ([#103799](https://github.com/kubernetes/kubernetes/pull/103799), [@jyz0309](https://github.com/jyz0309)) [SIG API Machinery, Cluster Lifecycle and Instrumentation] 3844 - Kubeadm: remove the --port flag from the manifest for the kube-controller-manager since the flag has been a NO-OP since 1.22 and insecure serving was removed for the component. ([#104157](https://github.com/kubernetes/kubernetes/pull/104157), [@knight42](https://github.com/knight42)) [SIG Cluster Lifecycle] 3845 3846 ### API Change 3847 3848 - CSIDriver.Spec.StorageCapacity can now be modified. ([#101789](https://github.com/kubernetes/kubernetes/pull/101789), [@pohly](https://github.com/pohly)) [SIG Storage] 3849 - Kube-apiserver: The `rbac.authorization.k8s.io/v1alpha1` API version is removed; use the `rbac.authorization.k8s.io/v1` API, available since v1.8. The `scheduling.k8s.io/v1alpha1` API version is removed; use the `scheduling.k8s.io/v1` API, available since v1.14. ([#104248](https://github.com/kubernetes/kubernetes/pull/104248), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Auth, Network and Testing] 3850 - Kube-controller-manager supports '--concurrent-ephemeralvolume-syncs' flag to set the number of ephemeral volume controller workers. ([#102981](https://github.com/kubernetes/kubernetes/pull/102981), [@SataQiu](https://github.com/SataQiu)) [SIG API Machinery and Apps] 3851 3852 ### Feature 3853 3854 - Adding support for multiple --from-env-file flags ([#101646](https://github.com/kubernetes/kubernetes/pull/101646), [@lauchokyip](https://github.com/lauchokyip)) [SIG CLI] 3855 - All folks to build kubernetes with a custom kube-cross image ([#104185](https://github.com/kubernetes/kubernetes/pull/104185), [@dims](https://github.com/dims)) [SIG Release and Testing] 3856 - Allow node expansion of local volumes ([#102886](https://github.com/kubernetes/kubernetes/pull/102886), [@gnufied](https://github.com/gnufied)) [SIG Storage and Testing] 3857 - Client-go event library allows customizing spam filtering function. 3858 It is now possible to override `SpamKeyFunc`, which is used by event filtering to detect spam in the events. ([#103918](https://github.com/kubernetes/kubernetes/pull/103918), [@olagacek](https://github.com/olagacek)) [SIG API Machinery and Instrumentation] 3859 - Constants/variables from k8s.io for STABLE metrics is now supported ([#103654](https://github.com/kubernetes/kubernetes/pull/103654), [@coffeepac](https://github.com/coffeepac)) [SIG Auth, Instrumentation, Node and Testing] 3860 - Display Labels when kubectl describe ingress ([#103894](https://github.com/kubernetes/kubernetes/pull/103894), [@kabab](https://github.com/kabab)) [SIG CLI] 3861 - Expose a `NewUnstructuredExtractor` from apply configurations `meta/v1` package that enables extracting objects into unstructured apply configurations ([#103564](https://github.com/kubernetes/kubernetes/pull/103564), [@kevindelgado](https://github.com/kevindelgado)) [SIG API Machinery, Cluster Lifecycle, Release and Testing] 3862 - Introduce a feature gate DisableKubeletCloudCredentialProviders which allows disabling the in-tree kubelet credential providers. 3863 3864 The DisableKubeletCloudCredentialProviders FeatureGate is currently in Alpha, which means is currently disabled by default. Once the FeatureGate moves to beta, in-tree credential providers will be disabled by default, and users will need to migrate to using external credential providers. ([#102507](https://github.com/kubernetes/kubernetes/pull/102507), [@ostrain](https://github.com/ostrain)) [SIG Cloud Provider] 3865 - Introduces a new metric: admission_webhook_request_total with the following labels: name (string) - the webhook name, type (string) - the admission type, operation (string) - the requested verb, code (int) - the HTTP status code, rejected (bool) - whether the request was rejected, namespace (string) - the namespace of the requested resource. ([#103162](https://github.com/kubernetes/kubernetes/pull/103162), [@rmoriar1](https://github.com/rmoriar1)) [SIG API Machinery and Instrumentation] 3866 - Kube-up.sh installs csi-proxy v1.0.1-gke.0 ([#104426](https://github.com/kubernetes/kubernetes/pull/104426), [@mauriciopoppe](https://github.com/mauriciopoppe)) [SIG Cloud Provider, Storage and Windows] 3867 - Kubeadm: add support for dry running "kubeadm join". The new flag "kubeadm join --dry-run" is similar to the existing flag for "kubeadm init/upgrade" and allows you to see what changes would be applied. ([#103027](https://github.com/kubernetes/kubernetes/pull/103027), [@Haleygo](https://github.com/Haleygo)) [SIG Cluster Lifecycle] 3868 - Kubernetes is now built with Golang 1.16.7 ([#104199](https://github.com/kubernetes/kubernetes/pull/104199), [@cpanato](https://github.com/cpanato)) [SIG Cloud Provider, Instrumentation, Release and Testing] 3869 - The ServiceAccountIssuerDiscovery feature gate is removed. It reached GA in Kubernetes 1.21. ([#103685](https://github.com/kubernetes/kubernetes/pull/103685), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery and Auth] 3870 - Updated Cluster Autosaler to version 1.22.0. Release notes: https://github.com/kubernetes/autoscaler/releases/tag/cluster-autoscaler-1.22.0 ([#104293](https://github.com/kubernetes/kubernetes/pull/104293), [@x13n](https://github.com/x13n)) [SIG Autoscaling and Cloud Provider] 3871 - Updates the following images to pick up CVE fixes: 3872 - debian to v1.9.0 3873 - debian-iptables to v1.6.6 3874 - setcap to v2.0.4 ([#104142](https://github.com/kubernetes/kubernetes/pull/104142), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery, Release and Testing] 3875 3876 ### Documentation 3877 3878 - Update description of --audit-log-maxbackup to describe behavior when value = 0 ([#103843](https://github.com/kubernetes/kubernetes/pull/103843), [@Arkessler](https://github.com/Arkessler)) [SIG API Machinery] 3879 3880 ### Bug or Regression 3881 3882 - 1. Changes json representation for a conflicted taint to Key=Effect when a conflicted taint occurs in kubectl taint. ([#104011](https://github.com/kubernetes/kubernetes/pull/104011), [@manugupt1](https://github.com/manugupt1)) [SIG CLI] 3883 - A new server run option 'shutdown-send-retry-after' has been introduced. If true the HTTP Server 3884 will continue listening until all non longrunning request(s) in flight have been drained, during this window all 3885 incoming requests will be rejected with a status code 429 and a 'Retry-After' response header. ([#101257](https://github.com/kubernetes/kubernetes/pull/101257), [@tkashem](https://github.com/tkashem)) [SIG API Machinery] 3886 - Adds Kubernetes Events to the Kubelet Graceful Shutdown feature ([#101081](https://github.com/kubernetes/kubernetes/pull/101081), [@rphillips](https://github.com/rphillips)) [SIG Node] 3887 - CA, certificate and key bundles for the generic-apiserver based servers will be reloaded immediately after the files are changed. ([#104102](https://github.com/kubernetes/kubernetes/pull/104102), [@tnqn](https://github.com/tnqn)) [SIG API Machinery and Testing] 3888 - Fix kube-apiserver metric reporting for the deprecated watch path of /api/<version>/watch/... ([#104161](https://github.com/kubernetes/kubernetes/pull/104161), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery and Instrumentation] 3889 - Fix: skip case sensitivity when checking Azure NSG rules ([#104384](https://github.com/kubernetes/kubernetes/pull/104384), [@feiskyer](https://github.com/feiskyer)) [SIG Cloud Provider] 3890 - Fixed an issue which didn't append OS's environment variables with the one provided in Credential Provider Config file, which may lead to failed execution of external credential provider binary. 3891 See https://github.com/kubernetes/kubernetes/issues/102750 ([#103231](https://github.com/kubernetes/kubernetes/pull/103231), [@n4j](https://github.com/n4j)) [SIG Auth and Node] 3892 - Fixed architecture within manifest for non `amd64` etcd images. ([#104116](https://github.com/kubernetes/kubernetes/pull/104116), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery] 3893 - Fixed bug where kubectl would emit duplicate warning messages for flag names that contain an underscore and recommend using a nonexistent flag in some cases ([#103852](https://github.com/kubernetes/kubernetes/pull/103852), [@brianpursley](https://github.com/brianpursley)) [SIG CLI and Cluster Lifecycle] 3894 - Graceful node shutdown, allow the actual inhibit delay to be greater than the expected inhibit delay ([#103137](https://github.com/kubernetes/kubernetes/pull/103137), [@wzshiming](https://github.com/wzshiming)) [SIG Node] 3895 - Kube-apiserver: Avoids unnecessary repeated calls to admission webhooks that reject an update or delete request. ([#104182](https://github.com/kubernetes/kubernetes/pull/104182), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] 3896 - Kube-proxy: delete stale conntrack UDP entries for loadbalancer ingress IP. ([#104009](https://github.com/kubernetes/kubernetes/pull/104009), [@aojea](https://github.com/aojea)) [SIG Network] 3897 - Kubeadm: When adding an etcd peer to an existing cluster, if an error is returned indicating the peer has already been added, this is accepted and a ListMembers call is used instead to return the existing cluster. This helps diminish the exponential backoff when the first AddMember call times out, while still retaining a similar performance when the peer had already been added from a previous call. ([#104134](https://github.com/kubernetes/kubernetes/pull/104134), [@ihgann](https://github.com/ihgann)) [SIG Cluster Lifecycle] 3898 - Pass additional flags to subpath mount to avoid flakes in certain conditions ([#104253](https://github.com/kubernetes/kubernetes/pull/104253), [@mauriciopoppe](https://github.com/mauriciopoppe)) [SIG Storage] 3899 - Update Go used to build migrate script in etcd image to v1.16.7 ([#104301](https://github.com/kubernetes/kubernetes/pull/104301), [@serathius](https://github.com/serathius)) [SIG API Machinery and Release] 3900 3901 ### Other (Cleanup or Flake) 3902 3903 - Deprecate apiserver_longrunning_gauge and apiserver_register_watchers in 1.23.0 ([#103793](https://github.com/kubernetes/kubernetes/pull/103793), [@yan-lgtm](https://github.com/yan-lgtm)) [SIG API Machinery, Cluster Lifecycle and Instrumentation] 3904 - Kube-apiserver: sets an upper-bound on the lifetime of idle keep-alive connections and time to read the headers of incoming requests ([#103958](https://github.com/kubernetes/kubernetes/pull/103958), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Node] 3905 - Kubeadm: external etcd endpoints passed in the ClusterConfiguration that have Unicode characters are no longer IDNA encoded (converted to Punycode). They are now just URL encoded as per Go's implementation of RFC-3986, have duplicate "/" removed from the URL paths, and passed like that directly to the kube-apiserver --etcd-servers flag. If you have etcd endpoints that have Unicode characters, it is advisable to encode them in advance with tooling that is fully IDNA compliant. If you don't do that, the Go standard library (used in k8s and etcd) would do it for you when making requests to the endpoints. ([#103801](https://github.com/kubernetes/kubernetes/pull/103801), [@gkarthiks](https://github.com/gkarthiks)) [SIG Cluster Lifecycle] 3906 - Kubeadm: update references to legacy artifacts locations, the 'ci-cross' prefix has been removed from the version match as it does not exist in the new 'gs://k8s-release-dev' bucket ([#103813](https://github.com/kubernetes/kubernetes/pull/103813), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] 3907 - Migratecmd/kube-proxy/app logs to structured logging ([#98913](https://github.com/kubernetes/kubernetes/pull/98913), [@yxxhero](https://github.com/yxxhero)) [SIG Network] 3908 - Surface warning when users don't set propagationPolicy for jobs while deleting ([#104080](https://github.com/kubernetes/kubernetes/pull/104080), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Apps] 3909 - The AllowInsecureBackendProxy feature gate is removed. It reached GA in Kubernetes 1.21. ([#103796](https://github.com/kubernetes/kubernetes/pull/103796), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery] 3910 - The `StartupProbe` feature gate that is GA since v1.20 is unconditionally enabled, and can no longer be specified via the `--feature-gates` argument. ([#104168](https://github.com/kubernetes/kubernetes/pull/104168), [@ialidzhikov](https://github.com/ialidzhikov)) [SIG Node] 3911 - The apiserver exposes 4 new metrics that allow to track the status of the Service CIDRs allocations: 3912 - current number of available IPs per Service CIDR 3913 - current number of used IPs per Service CIDR 3914 - total number of allocation per Service CIDR 3915 - total number of allocation errors per ServiceCIDR ([#104119](https://github.com/kubernetes/kubernetes/pull/104119), [@aojea](https://github.com/aojea)) [SIG Apps, Instrumentation and Network] 3916 - The flag `--deployment-controller-sync-period` has no effect now, deprecate it and will be removed in v1.24. ([#103538](https://github.com/kubernetes/kubernetes/pull/103538), [@Pingan2017](https://github.com/Pingan2017)) [SIG Apps] 3917 - Troubleshooting: informers log handlers that take more than 100 milliseconds to process an object if the DeltaFIFO queue starts to grow beyond 10 elements. ([#103917](https://github.com/kubernetes/kubernetes/pull/103917), [@aojea](https://github.com/aojea)) [SIG API Machinery] 3918 - Update cri-tools dependency to v1.22.0 ([#104430](https://github.com/kubernetes/kubernetes/pull/104430), [@saschagrunert](https://github.com/saschagrunert)) [SIG Cloud Provider and Node] 3919 - ``gcr.io/kubernetes-e2e-test-images`` will no longer be used in E2E / CI testing, ``k8s.gcr.io/e2e-test-images`` will be used instead. ([#103724](https://github.com/kubernetes/kubernetes/pull/103724), [@claudiubelu](https://github.com/claudiubelu)) [SIG API Machinery and Testing] 3920 3921 ## Dependencies 3922 3923 ### Added 3924 - github.com/google/martian/v3: [v3.1.0](https://github.com/google/martian/v3/tree/v3.1.0) 3925 - github.com/kr/fs: [v0.1.0](https://github.com/kr/fs/tree/v0.1.0) 3926 - github.com/pkg/sftp: [v1.10.1](https://github.com/pkg/sftp/tree/v1.10.1) 3927 3928 ### Changed 3929 - cloud.google.com/go/bigquery: v1.4.0 → v1.8.0 3930 - cloud.google.com/go/storage: v1.6.0 → v1.10.0 3931 - cloud.google.com/go: v0.54.0 → v0.81.0 3932 - github.com/GoogleCloudPlatform/k8s-cloud-provider: [7901bc8 → ea6160c](https://github.com/GoogleCloudPlatform/k8s-cloud-provider/compare/7901bc8...ea6160c) 3933 - github.com/bketelsen/crypt: [5cbc8cc → v0.0.4](https://github.com/bketelsen/crypt/compare/5cbc8cc...v0.0.4) 3934 - github.com/golang/mock: [v1.4.4 → v1.5.0](https://github.com/golang/mock/compare/v1.4.4...v1.5.0) 3935 - github.com/google/pprof: [1ebb73c → cbba55b](https://github.com/google/pprof/compare/1ebb73c...cbba55b) 3936 - github.com/hashicorp/golang-lru: [v0.5.1 → v0.5.0](https://github.com/hashicorp/golang-lru/compare/v0.5.1...v0.5.0) 3937 - github.com/ianlancetaylor/demangle: [5e5cf60 → 28f6c0f](https://github.com/ianlancetaylor/demangle/compare/5e5cf60...28f6c0f) 3938 - github.com/magiconair/properties: [v1.8.1 → v1.8.5](https://github.com/magiconair/properties/compare/v1.8.1...v1.8.5) 3939 - github.com/mitchellh/go-homedir: [v1.1.0 → v1.0.0](https://github.com/mitchellh/go-homedir/compare/v1.1.0...v1.0.0) 3940 - github.com/mitchellh/mapstructure: [v1.1.2 → v1.4.1](https://github.com/mitchellh/mapstructure/compare/v1.1.2...v1.4.1) 3941 - github.com/pelletier/go-toml: [v1.2.0 → v1.9.3](https://github.com/pelletier/go-toml/compare/v1.2.0...v1.9.3) 3942 - github.com/prometheus/common: [v0.26.0 → v0.28.0](https://github.com/prometheus/common/compare/v0.26.0...v0.28.0) 3943 - github.com/spf13/afero: [v1.2.2 → v1.6.0](https://github.com/spf13/afero/compare/v1.2.2...v1.6.0) 3944 - github.com/spf13/cast: [v1.3.0 → v1.3.1](https://github.com/spf13/cast/compare/v1.3.0...v1.3.1) 3945 - github.com/spf13/cobra: [v1.1.3 → v1.2.1](https://github.com/spf13/cobra/compare/v1.1.3...v1.2.1) 3946 - github.com/spf13/jwalterweatherman: [v1.0.0 → v1.1.0](https://github.com/spf13/jwalterweatherman/compare/v1.0.0...v1.1.0) 3947 - github.com/spf13/viper: [v1.7.0 → v1.8.1](https://github.com/spf13/viper/compare/v1.7.0...v1.8.1) 3948 - go.opencensus.io: v0.22.3 → v0.23.0 3949 - golang.org/x/net: 37e1c6a → abc4532 3950 - golang.org/x/oauth2: bf48bf1 → f6687ab 3951 - google.golang.org/api: v0.20.0 → v0.46.0 3952 - google.golang.org/appengine: v1.6.5 → v1.6.7 3953 - gopkg.in/ini.v1: v1.51.0 → v1.62.0 3954 - honnef.co/go/tools: v0.0.1-2020.1.3 → v0.0.1-2020.1.4 3955 - k8s.io/gengo: b6c5ce2 → 485abfe 3956 - k8s.io/kube-openapi: 9528897 → 7fbd8d5 3957 - k8s.io/utils: 4b05e18 → efc7438 3958 3959 ### Removed 3960 - cloud.google.com/go/datastore: v1.1.0 3961 - cloud.google.com/go/pubsub: v1.2.0 3962 - github.com/alecthomas/units: [f65c72e](https://github.com/alecthomas/units/tree/f65c72e) 3963 - github.com/coreos/bbolt: [v1.3.2](https://github.com/coreos/bbolt/tree/v1.3.2) 3964 - github.com/coreos/etcd: [v3.3.13+incompatible](https://github.com/coreos/etcd/tree/v3.3.13) 3965 - github.com/coreos/go-systemd: [95778df](https://github.com/coreos/go-systemd/tree/95778df) 3966 - github.com/coreos/pkg: [399ea9e](https://github.com/coreos/pkg/tree/399ea9e) 3967 - github.com/dgrijalva/jwt-go: [v3.2.0+incompatible](https://github.com/dgrijalva/jwt-go/tree/v3.2.0) 3968 - github.com/google/martian: [v2.1.0+incompatible](https://github.com/google/martian/tree/v2.1.0) 3969 - github.com/jpillora/backoff: [v1.0.0](https://github.com/jpillora/backoff/tree/v1.0.0)