vitess.io/vitess@v0.16.2/changelog/13.0/13.0.0/summary.md (about) 1 ## Major Changes 2 3 ### Vitess now has native support for MySQL collations 4 5 When using the gen4 planner, Vitess is now capable of performing collation-aware string comparisons in the vtgates. This 6 improves the performance and reliability of several query plans that were previously relying on a debug-only SQL API in 7 MySQL to perform these comparisons remotely. It also enables new query plans that were previously not possible. 8 9 A full list of the supported collations can be 10 found [in the Vitess documentation](https://vitess.io/docs/13.0/user-guides/configuration-basic/collations/). 11 12 ### The native Evaluation engine in vtgate has been greatly improved 13 14 The SQL evaluation engine that runs inside the vtgates has been rewritten mostly from scratch to more closely match 15 MySQL's behavior. This allows Vitess to execute more parts of the query plans locally, and increases the complexity and 16 semantics of the SQL expressions that can be used to perform cross-shard queries. 17 18 ### vttablet -use_super_read_only flag now defaults to true 19 20 The default value used to be false. What this means is that during a failover, we will set `super_read_only` on database 21 flavors that support them (MySQL 5.7+ and Percona 5.7+). In addition, all Vitess-managed databases will be started 22 with `super-read-only` in the cnf file. It is expected that this change is safe and backwards-compatible. Anyone who is 23 relying on the current behavior should pass `-use_super_read_only=false` on the vttablet command line, and make sure 24 they are using a custom my.cnf instead of the one provided as the default by Vitess. 25 26 ### vtgate -buffer_implementation now defaults to keyspace_events 27 28 The default value used to be `healthcheck`. The new `keyspace_events` implementation has been tested in production with 29 good results and shows more consistent buffering behavior during PlannedReparentShard operations. The `keyspace_events` 30 implementation utilizes heuristics to detect additional cluster states where buffering is safe to perform, including 31 cases where the primary may be down. If there is a need to revert back to the previous buffer implementation, ensure 32 buffering is enabled in vtgate and pass the flag `-buffer_implementation=healthcheck`. 33 34 ### ddl_strategy: -postpone-completion flag 35 36 `ddl_strategy` (either `@@ddl_strategy` in VtGate or `-ddl_strategy` in `vtctlclient ApplySchema`) supports the 37 flag `-postpone-completion` 38 39 This flag indicates that the migration should not auto-complete. This applies for: 40 41 - any `CREATE TABLE` 42 - any `DROP TABLE` 43 - `ALTER` table in `online` strategy 44 - `ALTER` table in `gh-ost` strategy 45 46 Note that this flag is not supported for `pt-osc` strategy. 47 48 Behavior of migrations with this flag: 49 50 - an `ALTER` table begins, runs, but does not cut-over. 51 - `CREATE` or `DROP` migrations are silently not even scheduled 52 53 ### alter vitess_migration ... cleanup 54 55 A new query is supported: 56 57 ```sql 58 alter vitess_migration '9748c3b7_7fdb_11eb_ac2c_f875a4d24e90' cleanup 59 ``` 60 61 This query tells Vitess that a migration's artifacts are good to be cleaned up asap. This allows Vitess to free disk 62 resources sooner. As a reminder, once a migration's artifacts are cleaned up, the migration is no longer revertible. 63 64 ### alter vitess_migration ... complete 65 66 A new query is supported: 67 68 ```sql 69 alter vitess_migration '9748c3b7_7fdb_11eb_ac2c_f875a4d24e90' complete 70 ``` 71 72 This command indicates that a migration executed with `-postpone-completion` is good to complete. Behavior: 73 74 - For running `ALTER`s (`online` and `gh-ost`) which are ready to cut-over: cut-over imminently (though not immediately 75 - cut-over depends on polling interval, replication lag, etc) 76 - For running `ALTER`s (`online` and `gh-ost`) which are only partly through the migration: they will cut-over 77 automatically when they complete their work, as if `-postpone-completion` wasn't indicated 78 - For queued `CREATE` and `DROP` migrations: "unblock" them from being scheduled. They'll be scheduled at the scheduler' 79 s discretion. there is no guarantee that they will be scheduled to run immediately. 80 81 ### vtctl/vtctlclient ApplySchema: ALTER VITESS_MIGRATION 82 83 `vtctl ApplySchema` now supports `ALTER VITESS_MIGRATION ...` statements. Example: 84 85 ```shell 86 $ vtctl ApplySchema -skip_preflight -sql "alter vitess_migration '9748c3b7_7fdb_11eb_ac2c_f875a4d24e90' complete" commerce 87 ``` 88 89 ### vtctl/vtctlclient ApplySchema: allow zero in date 90 91 `vtctl/vtctlclient ApplySchema` now respects `-allow-zero-in-date` for `direct` strategy. For example, the following 92 statement is now accepted: 93 94 ```shell 95 vtctlclient ApplySchema -skip_preflight -ddl_strategy='direct -allow-zero-in-date' -sql "create table if not exists t2(id int primary key, dt datetime default '0000-00-00 00:00:00')" commerce 96 ``` 97 98 ### vtctl/vtctlclient ApplySchema -uuid_list 99 100 `vtctlient ApplySchema` now support a new optional `-uuid_list` flag. It is possible for the user to explicitly specify 101 the UUIDs for given migration(s). UUIDs must be in a specific format. If given, number of UUIDs must match the number of 102 DDL statements. Example: 103 104 ```shell 105 vtctlclient OnlineDDL ApplySchema -sql "drop table t1, drop table t2" -uuid_list "d08f0000_51c9_11ec_9cf2_0a43f95f28a3,d08f0001_51c9_11ec_9cf2_0a43f95f28a3" commerce 106 ``` 107 108 Vitess will assign each migration with given UUID in order of appearance. It is the user's responsibility to ensure 109 given UUIDs are globally unique. If the user submits a migration with an already existing UUID, that migration never 110 gets scheduled nor executed. 111 112 ### vtctl/vtctlclient ApplySchema -migration_context 113 114 `-migration_context` flag is synonymous to `-request_context`. Either will work. We will encourage use 115 of `-migration_context` as it is more consistent with output of `SHOW VITESS_MIGRATIONS ...` which includes 116 the `migration_context` column. 117 118 ### vtctl/vtctlclient OnlineDDL ... complete 119 120 Complementing the `alter vitess_migration ... complete` query, a migration can also be completed via `vtctl` 121 or `vtctlclient`: 122 123 ```shell 124 vtctlclient OnlineDDL <keyspace> complete <uuid> 125 ``` 126 127 For example: 128 129 ```shell 130 vtctlclient OnlineDDL commerce complete d08ffe6b_51c9_11ec_9cf2_0a43f95f28a3 131 ``` 132 133 ### vtctl/vtctlclient OnlineDDL -json 134 135 The command now accepts an optional `-json` flag. With this flag, the output is a valid JSON listing all columns and 136 rows. 137 138 ## vtadmin-web updated to node v16.13.0 (LTS) 139 140 Building vtadmin-web now requires node >= v16.13.0 (LTS). Upgrade notes are given 141 in https://github.com/vitessio/vitess/pull/9136. 142 143 ### PlannedReparentShard for cluster initialization 144 For setting up the cluster and electing a primary for the first time, `PlannedReparentShard` should be used 145 instead of `InitShardPrimary`. 146 147 `InitShardPrimary` is a forceful command and copies over the executed gtid set from the new primary to all the other replicas. So, if the user 148 isn't careful, it can lead to some replicas not being setup correctly and lead to errors in replication and recovery later. 149 `PlannedReparentShard` is a safer alternative and does not change the executed gtid set on the replicas forcefully. It is the preferred alternate to initialize 150 the cluster. 151 152 If using a custom `init_db.sql` that omits `SET sql_log_bin = 0`, then `InitShardPrimary` should still be used instead of `PlannedReparentShard`. 153 154 ### Durability Policy flag 155 A new flag has been added to vtctl, vtctld and vtworker binaries which allows the users to set the durability policies. 156 157 If semi-sync is not being used then `-durability_policy` should be set to `none`. This is also the default option. 158 159 If semi-sync is being used then `-durability_policy` should be set to `semi_sync` and `-enable_semi_sync` should be set in vttablets. 160 161 ## Incompatible Changes 162 ### Error message change when vttablet row limit exceeded: 163 * In previous Vitess versions, if the vttablet row limit (-queryserver-config-max-result-size) was exceeded, an error like: 164 ```shell 165 ERROR 10001 (HY000): target: unsharded.0.master: vttablet: rpc error: code = ResourceExhausted desc = Row count exceeded 10000 (errno 10001) (sqlstate HY000) ... 166 would be reported to the client. 167 ``` 168 To avoid confusion, the error mapping has been changed to report the error as similar to: 169 ```shell 170 ERROR 10001 (HY000): target: unsharded.0.primary: vttablet: rpc error: code = Aborted desc = Row count exceeded 10000 (errno 10001) (sqlstate HY000) ... 171 instead 172 ``` 173 * Because of this error code change, the vttablet metric: 174 `vttablet_errors{error_code="ABORTED"}` 175 will be incremented upon this type of error, instead of the previous metric: 176 `vttablet_errors{error_code="RESOURCE_EXHAUSTED"}` 177 * In addition, the vttablet error message logged is now different. 178 Previously, a (misleading; due to the PoolFull) error was logged: 179 ```shell 180 E0112 09:48:25.420641 278125 tabletserver.go:1368] PoolFull: Row count exceeded 10000 (errno 10001) (sqlstate HY000) ... 181 ``` 182 Post-change, a more accurate warning is logged instead: 183 ```shell 184 W0112 09:38:59.169264 35943 tabletserver.go:1503] Row count exceeded 10000 (errno 10001) (sqlstate HY000) ... 185 ``` 186 * If you were using -queryserver-config-terse-errors to redact some log messages containing bind vars in 13.0-SNAPSHOT, you should now instead enable -sanitize_log_messages which sanitizes all log messages containing sensitive info 187 188 ### Column types for textual queries now match MySQL's behavior 189 190 The column types for certain queries performed on a `vtgate` (most notably, those that `SELECT` system variables) 191 have been changed to match the types that would be returned if querying a MySQL instance directly: textual fields that 192 were previously returned as `VARBINARY` will now appear as `VARCHAR`. 193 194 This change should not have an impact on MySQL clients/connectors for statically typed programming languages, as these 195 clients coerce the returned rows into whatever types the user has requested, but clients for dynamic programming 196 languages may now start returning as "string" values that were previously returned as "bytes". 197 198 ## Deprecations 199 200 ### vtgate `-gateway_implementation` flag is deprecated (and ignored) 201 202 Support for `discoverygateway` is being dropped. `tabletgateway` is now the only supported implementation. Scripts using 203 this flag should be updated to remove the flag as it will be deleted in the next release. 204 205 ### web/vtctld2 is deprecated and can optionally be turned off 206 207 The vtctld2 web interface is no longer maintained and is planned for removal in Vitess 16. Motivation for this change and a roadmap to removing [the web/vtctld2 codebase](https://github.com/vitessio/vitess/tree/main/web/vtctld2) is given in https://github.com/vitessio/vitess/issues/9686. 208 209 Vitess operators can optionally disable the vtctld2 web interface ahead of time by calling the `vtctld` process with the flag `enable_vtctld_ui=false`. For more details on this flag, see https://github.com/vitessio/vitess/pull/9614.