vitess.io/vitess@v0.16.2/changelog/12.0/12.0.0/release_notes.md (about) 1 # Release of Vitess v12.0.0 2 ## Major Changes 3 4 This release includes the following major changes or new features. 5 6 ### Inclusive Naming 7 A number of CLI commands and vttablet RPCs have been deprecated as we move from `master` to `primary`. 8 All functionality is backwards compatible except as noted under Incompatible Changes. 9 **Deprecated commands and flags will be removed in the next release (13.0). 10 Deprecated vttablet RPCs will be removed in the subsequent release (14.0).** 11 12 ### Gen4 Planner 13 The newest version of the query planner, `Gen4`, becomes an experimental feature as part of this release. 14 While `Gen4` has been under development for a few release cycles, we have now reached parity with its predecessor, `v3`. 15 16 To use `Gen4`, VTGate's `-planner_version` flag needs to be set to `Gen4Fallback`. 17 18 ### Query Buffering during failovers 19 In order to support buffering during resharding cutovers in addition to primary failovers, a new implementation 20 of query buffering has been added. 21 This is considered experimental. To enable it the flag `buffer_implementation` should be set to `keyspace_events`. 22 The existing implementation (flag value `healthcheck`) will be deprecated in a future release. 23 24 ## Known Issues 25 26 - A critical vulnerability CVE-2021-44228 in the Apache Log4j logging library was disclosed on Dec 9 2021. 27 The project provided release `2.15.0` with a patch that mitigates the impact of this CVE. It was quickly found that the initial patch was insufficient, and additional CVEs 28 CVE-2021-45046 and CVE-2021-44832 followed. 29 These have been fixed in release `2.17.1`. This release of Vitess, `v12.0.0`, uses a version of Log4j below `2.17.1`, for this reason, we encourage you to use version `v12.0.3` instead, to benefit from the vulnerability patches. 30 31 - An issue where the value of the -force flag is used instead of -keep_data flag's value in v2 vreplication workflows (#9174) is known to be present in this release. A workaround is available in the description of issue #9174. This issue is fixed in release >= `v12.0.1`. 32 33 ## Incompatible Changes 34 35 ### CLI command output 36 Wherever CLI commands produced `master` or `MASTER` for tablet type, they now produce `primary` or `PRIMARY`. 37 Scripts and tools that depend on parsing command output will need to change. 38 39 Example: 40 ```sh 41 $ vtctlclient -server localhost:15999 ListAllTablets 42 zone1-0000000100 sourcekeyspace 0 primary 192.168.0.134:15100 192.168.0.134:17100 [] 2021-09-24T01:12:00Z 43 zone1-0000000101 sourcekeyspace 0 rdonly 192.168.0.134:15101 192.168.0.134:17101 [] <null> 44 zone1-0000000102 sourcekeyspace 0 replica 192.168.0.134:15102 192.168.0.134:17102 [] <null> 45 zone1-0000000103 sourcekeyspace 0 rdonly 192.168.0.134:15103 192.168.0.134:17103 [] <null> 46 ``` 47 48 ## Default Behaviour Changes 49 50 ### Enabling reserved connections 51 Use of reserved connections is controlled by the vtgate flag `-enable_system_settings`. This flag has in the past defaulted to `false` (or `OFF`) in release versions (i.e. x.0 and x.0.y) of Vitess, and to `true` (or `ON`) in development versions. 52 53 From Vitess 12.0.0 onwards, it defaults to ***true*** in all release and development versions. You can read more about this [here](https://github.com/vitessio/vitess/issues/9125). Hence you should specify this flag explicitly, so that you are sure whether it is enabled or not, regardless of which Vitess release/build/version you are running. 54 55 If you have reserved connections disabled, you will get the ***old*** Vitess behavior: where most system settings (e.g. `sql_mode`) are just silently ignored by Vitess. In situations where you know your backend MySQL defaults are acceptable, this may be the correct setting to ensure the best possible performance of the vttablet <-> MySQL connection pools. This comes down to a trade-off between compatibility and performance/scalability. You should also review [this section](https://vitess.io/docs/reference/query-serving/reserved-conn/#number-of-vttablet---mysql-connections) of the documentation when deciding whether or not to enable reserved connections. 56 57 ## Deprecations 58 59 ### CLI commands 60 `VExec` is deprecated and removed. All Online DDL commands should be run through `OnlineDDL`. 61 62 `OnlineDDL revert` is deprecated. Use `REVERT VITESS_MIGRATION '...'` SQL command either via `ApplySchema` or via `vtgate`. 63 64 `InitShardMaster` is deprecated, use `InitShardPrimary` instead. 65 66 `SetShardIsMasterServing` is deprecated, use `SetShardIsPrimaryServing` instead. 67 68 Various command flags have been deprecated and new variants provided. 69 * `DeleteTablet` flag `allow_master` replaced with `allow_primary` 70 * `PlannedReparentShard` flag `avoid_master` replaced with `avoid_tablet` 71 * `PlannedReparentShard` flag `new_master` replaced with `new_primary` 72 * `BackupShard` flag `allow_master` replaced with `allow_primary` 73 * `Backup` flag `allow_master` replaced with `allow_primary` 74 * `EmergencyReparentShard` flag `new_master` replaced with `new_primary` 75 * `ReloadSchemeShard` flag `include_master` replaced with `include_primary` 76 * `ReloadSchemaKeyspace` flag `include_master` replaced with `include_primary` 77 * `ValidateSchemaKeyspace` flag `skip-no-master` replaced with `skip-no-primary` 78 79 ## Minor Changes 80 81 ### Query Serving 82 * Add `SHOW VITESS_REPLICATION_STATUS` Query Support #8900 83 * The new `SHOW VITESS_REPLICATION_STATUS` command/query shows the MySQL replica/replication (not `vreplication`) health for the vitess deployment. 84 85 ## Governance 86 * Added @GuptaManan100 to maintainers #8833 87 88 ------------ 89 ## Changelog 90 91 ### Bug fixes 92 #### Build/CI 93 * Update golang/x dependencies #8688 94 * tests: use AtomicInt32 instead of int to fix races #8696 95 #### Cluster management 96 * Use consul lock properly #8310 97 * Trivial: mysqlctl reinit_config -h was not showing help #8369 98 * Port #8422 to main branch #8745 99 * [vtctl] Add missing call to `flag.Parse` in `commandGetRoutingRules` #8795 100 * Fix for padding in OrderAndCheckPartitions #8873 101 * tabletmanager: correct implementations of deprecated RPCs for backwards compatibility #9059 102 #### General 103 * Avoid some pollution of CLI argument namespace across vtgate and vttablet #9026 104 #### Observability 105 * schema engine: reset table size stats when a table is dropped #8634 106 #### Query Serving 107 * Fix for function calls in DEFAULT value of CREATE TABLE statement in main #8477 108 * Fixing multiple issues related to onlineddl/lifecycle #8500 109 * boolean values should not be parenthesised in default clause #8502 110 * mysql/server: abort connection if secure transport is required #8504 111 * Clear only the current shard's schemacopy rows during schema reload #8519 112 * go/mysql/auth: fix error message for wrong auth method. #8525 113 * Fix handshake protocol with MySQL client #8537 114 * Copy parameter data from byte buffer in COM_STMT_SEND_LONG_DATA #8562 115 * Negative float default #8587 116 * [vtexplain] prevent panic on show tables/show full tables #8601 117 * OnlineDDL: better scheduling/cancellation logic #8603 118 * Fix parsing of FIRST clause in ALTER TABLE #8614 119 * onlineddl Executor: build schema with DBA user #8624 120 * Handle subquery merging with references correctly #8662 121 * Fixing error on deletes from owning table for a lookup being populated by CreateLookupVindex after a SwitchWrites #8701 122 * Fixing a panic in vtgate with OLAP mode #8722 123 * default to primary tablet if not set in VStream api #8755 124 * Fix bug in parsing of version comments #8770 125 * Proper merge of the SysTableTableName fields when joining two routes #8771 126 * fix parsing of MySQL Server Version flag #8791 127 * Fix typo in pool config syntax. Fixes #8819 #8820 128 * VtGate schema tracker to use provided user #8857 129 * fix merging dba query as a subquery both in v3 and gen4 #8871 130 * sql_select_limit support #9027 131 * [12.0] Handle sql_mode setting to compare modes in any order #9041 132 * [12.0] Fix for schema tracker not recognizing RENAME for schema reloading #9043 133 * [12.0] Fix savepoint support with reserved connections #9047 134 * [12.0] Gen4: Handle single column vindex correctly with multi column #9061 135 #### VReplication 136 * MoveTables: don't create unnecessary streams on the target for non-intersecting sources and targets #8090 137 * VDiff: Add BIT datatype to list of byte comparable types #8401 138 * Fix vreplication error metric #8483 139 * Return from throttler goroutine if context is cancelled to prevent goroutine leaks #8489 140 * Fix VReplication logging to file and db #8521 141 * Fix passing the wrong cell/cells variable to CreateLookupVindex #8590 142 * Refresh SrvVSchema after an ExternalizeVindex: was missing #8670 143 * Fixing missing argument in function call #8692 144 * Fix how we identify MySQL generated columns #8763 145 #### VTAdmin 146 * [vtadmin-web] Set Backup status indicators to the correct colours #8410 147 * [vtadmin] Fix bad copy-paste in pool config flag parsing #8518 148 #### vtctl 149 * Print actual current datetime for vtctl dry run commands #8778 150 #### vttestserver 151 * Remove lingering mysqld sock files on vttestserver container startup #8463 152 * Fixed filename for the configuration file in Vttestserver #8809 153 ### CI/Build 154 #### Build/CI 155 * Update release process #7759 156 * [ci] Add `errcheck` to golangci-lint #7996 157 * gomod: do not replace GRPC #8416 158 * consolidation: Fix flaky test #8417 159 * Small build improvements #8418 160 * proto: upgrade vtprotobuf version #8454 161 * Automatically use the latest tag of Vitess for cluster upgrade E2E test #8471 162 * Addition of known issues to release notes #8482 163 * Fix Cluster 14 flakiness #8494 164 * add question on backporting back to PR template #8541 165 * Enable GitHub Actions concurrency feature #8546 166 * Updated Makefile do_release script to include godoc steps #8550 167 * Changing codahale/hdrhistogram import path #8637 168 * Upgrade to Go 1.17 #8640 169 * Fixes for reparent endtoend test flakiness #8642 170 * hooks: remove govet because it already runs as part of golangci-lint #8674 171 * Enhancement of the release instructions #8790 172 * Remove consul-api usage in favor of official consul/api #8794 173 * Enhancement of the release notes generation #8877 174 #### Governance 175 * update MAINTAINERS and CODEOWNERS for deepthi, pH14 and vmg #8675 176 #### Query Serving 177 * Cleanup: import of gh-ost test suite is complete #8459 178 #### VReplication 179 * VReplication support for non-PRIMARY KEY (Online DDL context) #8364 180 ### Documentation 181 #### Build/CI 182 * maintainers: add Messaging as area of expertise #8578 183 #### Cluster management 184 * Enhance k8stopo flag documentation #8464 185 * Update some vtctl/vtctlclient command help #8560 186 * Trivial: fix the GenerateShardRanges vtctl[client] help #8586 187 #### Examples 188 * examples: update README to use new vreplication command syntax #8825 189 #### Governance 190 * update governance #8609 191 #### Query Serving 192 * vtexplain examples #8652 193 #### VReplication 194 * vtctl help/usage update #8879 195 #### VTAdmin 196 * [vtadmin] Remove outdated commands from vtadmin-web README #8496 197 * Initial pass at some vtadmin docs #8526 198 ### Enhancement 199 #### Backup and Restore 200 * Add Support for Restoring Specific Backups #8824 201 #### Build/CI 202 * Initial support for stress testing in end-to-end tests #8406 203 * CI: Remove mariadb101 which is well past eol and no longer available in distributions #8446 204 * Adding GitHub Self Hosted Runner tests #8721 205 #### Cluster management 206 * Vitess mixin improvements #7499 207 * Expose topo_consul_lock_session_checks to allow customized consul session check #8298 208 * Handle lock release with SIGHUP in VTGR #8472 209 * Enhance PRS error message #8529 210 * srvtopo: resilient watcher #8583 211 * Add `-restart_before_backup` parameter for vtbackup #8608 212 * srvtopo: allow unwatching from watch callbacks #8633 213 * introduced cluster_operation as a new error code in vtrpc #8646 214 * servenv: add `--onclose_timeout` flag #8651 215 * Improve determinism of bootstrap #8840 216 * OnlineDDL in v12: remove legacy code deprecated in earlier version #8972 217 #### Observability 218 * [vtadmin] cluster debug pages #8427 219 * Export memstats for statsd backend #8777 220 #### Query Serving 221 * [tablet, queryrules] Extend query rules to check comments #8233 222 * Add "show global status" support #8344 223 * added batch lookup param to lookup vindex #8398 224 * Online DDL/VReplication: able to read from replica #8405 225 * multishard autocommit should work for bypass routing #8428 226 * Add "no-scatter" flag to prohibit the use of scatter queries #8439 227 * OnlineDDL: -skip-topo is always 'true' #8450 228 * Add `rows` as keyword in sqlparser #8467 229 * Periodic update of gh-ost binary #8470 230 * fix when show tables record too much error log "Got unhandled packet.." #8478 231 * SHOW VITESS_MIGRATION '...' LOGS, retain logs for 24 hours #8493 232 * NativeDDL: analyzing added&removed unique keys #8495 233 * Refactor authentication server plugin mechanism #8503 234 * Update to planetscale/tengo v0.10.1-ps.v4 #8516 235 * query serving to continue when topo server restarts - main #8534 236 * support grpc reflection on APIs #8551 237 * Customized CreateLookUpVindex to be Compatible with non-consistent Lookup Vindex #8570 238 * Add additional options for configuring SSL modes as a client #8588 239 * Fail plan for unsupported aggregate function #8593 240 * gateway: use keyspace events when buffering requests #8700 241 * Add optional query annotations, i.e. prefix SQL comments indicating the #8783 242 * Slight improvement of the ACL error message #8805 243 * Changed parsing for unions #8821 244 * Use vt_dba user for online schema migration cutover phase #8836 245 * Add parsing support for column list with subquery alias #8884 246 * Add planner-version flag to vtexplain #8979 247 * [Backport 12.0]: Add planner-version flag to vtexplain #9003 248 #### TabletManager 249 * Allow min TLS version for tablet to mysqld conns #8757 250 #### VReplication 251 * VReplication: Add ability to tag workflows #8388 252 * Ignore SBR statements from pt-table-checksum #8396 253 * Tablet Picker: add metric to record lack of available tablets #8403 254 * VReplication: support different keys on source and target tables (different covered columns) #8423 255 * VStream API: handle reparenting and unhealthy tablets #8445 256 * Change local example to use v2 vreplication flows and change default to v2 from v1 #8553 257 * Customized CreateLookUpVindex to Include a Flag to Continue Vreplication After Backfill with an Owner Provided #8572 258 * Update MoveTables CLI help to reflect change we made at the beginning of #8597 259 * VStreamer Field/Row Events: add Keyspace/Shard #8598 260 * VStream API: Add flag to stop streaming on a reshard #8628 261 #### VTAdmin 262 * [vtadmin] cluster rpc pools #8421 263 * Add support for passing custom interceptors to vtadmin grpcserver #8507 264 ### Feature Request 265 #### Cluster management 266 * [vtctldserver] Migrate remaining ServingGraph rpcs to VtctldServer #8249 267 * [grpctmclient] Add support for (bounded) per-host connection reuse #8368 268 * VTGR: Vitess + MySQL group replication #8387 269 * Add isActive flag to vtgr to support multi-cell topology #8780 270 * [vtctld] Add `SetWritable`, `StartReplication` and `StopReplication` rpcs #8816 271 * [vtctld] sleep/ping tablets #8826 272 * [vtctld] migrate more util rpcs #8843 273 * [vtctld] migrate validator rpcs #8849 274 * [vtctld] localvtctldclient #8882 275 * [vtctldserver] Migrate RunHealthCheck #8892 276 * [vtctl] run new commands as standalone binary #8893 277 #### VTAdmin 278 * [vtadmin] [experimental] add per-api RBAC implementation #8515 279 * [vtadmin] shard replication positions #8775 280 * [vtadmin] GetVtctlds #8792 281 ### Internal Cleanup 282 #### Build/CI 283 * Bump aws-sdk-go to v1.34.2 #8632 284 * Upgrade consul api: `go get github.com/hashicorp/consul/api@v1.10.1` #8784 285 #### Cluster management 286 * topodata: remove deprecated field 'ServedTypes' #8566 287 * Upgrade to k8s 1.18 client #8762 288 * [vtctl] command deprecations #8967 289 #### Deployments 290 * Remove the deprecated Helm charts and related code #8868 291 #### Examples 292 * Bump ini from 1.3.5 to 1.3.8 in /vitess-mixin/e2e #8823 293 * backport operator updates to v12 branch . #9057 294 #### General 295 * Migrate k8s topo CRD to v1 api (v12 branch) #9046 296 #### Governance 297 * fix copyright #8611 298 #### Query Serving 299 * Adds flag to vttablet to disallow online DDL statements #8433 300 * Remove vtrpc.LegacyErrorCode #8456 301 * Allow for configuration of the minimal TLS version #8460 302 * engine: allow retrying partial primitives #8727 303 * srvtopo: expose WatchSrvKeyspace #8752 304 * Clean up Primitive interface implementations #8901 305 #### VReplication 306 * LegacySplitCloneWorker is no longer used in any code paths #8867 307 #### VTAdmin 308 * [vtadmin] Add Options struct to wrap grpc/http options #8461 309 * Update vtadmin local scripts to enable basic rbac #8801 310 #### vtctl 311 * Correctly identify backup timestamp var as a string #8891 312 ### Other 313 #### Examples 314 * Improve the Docker local and compose examples #8685 315 ### Performance 316 #### Cluster management 317 * throttler: don't allocate any resources unless it is actually enabled #8643 318 #### Query Serving 319 * cache: track memory usage more closely #8804 320 #### VTAdmin 321 * Use `fmt.Fprintf` instead of `fmt.Sprintf` #8922 322 ### Testing 323 #### Cluster management 324 * Alter tests to reuse cluster components #8276 325 * Adds some more orchestrator tests to vtorc #8535 326 #### Query Serving 327 * increase conn killer check to double the tx timeout value #8649 328 * Added UNION testcases and auxiliary code for running tests #8797 329 * fixed regression in v3 for grouping by integer functions #8856 330 #### VTAdmin 331 * [vtadmin] Add a vtctld Dialer unit test #8455 332 333 334 The release includes 1351 commits (excluding merges) 335 336 Thanks to all our contributors: @Anders-PlanetScale, @GuptaManan100, @Johnny-Three, @Juneezee, @ajm188, @aquarapid, @askdba, @bnu0, @carsonoid, @choo-stripe, @dbussink, @dctrwatson, @deepthi, @demmer, @dependabot[bot], @derekperkins, @doeg, @eeSeeGee, @falun, @fatih, @frouioui, @hallaroo, @harshit-gangal, @ilikeorangutans, @mattlord, @rafael, @ritwizsinha, @rohit-nayak-ps, @sahutd, @shlomi-noach, @sonne5, @systay, @tdakkota, @tokikanno, @utk9, @vmg