vitess.io/vitess@v0.16.2/changelog/13.0/13.0.0/summary.md (about)

     1  ## Major Changes
     2  
     3  ### Vitess now has native support for MySQL collations
     4  
     5  When using the gen4 planner, Vitess is now capable of performing collation-aware string comparisons in the vtgates. This
     6  improves the performance and reliability of several query plans that were previously relying on a debug-only SQL API in
     7  MySQL to perform these comparisons remotely. It also enables new query plans that were previously not possible.
     8  
     9  A full list of the supported collations can be
    10  found [in the Vitess documentation](https://vitess.io/docs/13.0/user-guides/configuration-basic/collations/).
    11  
    12  ### The native Evaluation engine in vtgate has been greatly improved
    13  
    14  The SQL evaluation engine that runs inside the vtgates has been rewritten mostly from scratch to more closely match
    15  MySQL's behavior. This allows Vitess to execute more parts of the query plans locally, and increases the complexity and
    16  semantics of the SQL expressions that can be used to perform cross-shard queries.
    17  
    18  ### vttablet -use_super_read_only flag now defaults to true
    19  
    20  The default value used to be false. What this means is that during a failover, we will set `super_read_only` on database
    21  flavors that support them (MySQL 5.7+ and Percona 5.7+). In addition, all Vitess-managed databases will be started
    22  with `super-read-only` in the cnf file. It is expected that this change is safe and backwards-compatible. Anyone who is
    23  relying on the current behavior should pass `-use_super_read_only=false` on the vttablet command line, and make sure
    24  they are using a custom my.cnf instead of the one provided as the default by Vitess.
    25  
    26  ### vtgate -buffer_implementation now defaults to keyspace_events
    27  
    28  The default value used to be `healthcheck`. The new `keyspace_events` implementation has been tested in production with
    29  good results and shows more consistent buffering behavior during PlannedReparentShard operations. The `keyspace_events`
    30  implementation utilizes heuristics to detect additional cluster states where buffering is safe to perform, including
    31  cases where the primary may be down. If there is a need to revert back to the previous buffer implementation, ensure
    32  buffering is enabled in vtgate and pass the flag `-buffer_implementation=healthcheck`.
    33  
    34  ### ddl_strategy: -postpone-completion flag
    35  
    36  `ddl_strategy` (either `@@ddl_strategy` in VtGate or `-ddl_strategy` in `vtctlclient ApplySchema`) supports the
    37  flag `-postpone-completion`
    38  
    39  This flag indicates that the migration should not auto-complete. This applies for:
    40  
    41  - any `CREATE TABLE`
    42  - any `DROP TABLE`
    43  - `ALTER` table in `online` strategy
    44  - `ALTER` table in `gh-ost` strategy
    45  
    46  Note that this flag is not supported for `pt-osc` strategy.
    47  
    48  Behavior of migrations with this flag:
    49  
    50  - an `ALTER` table begins, runs, but does not cut-over.
    51  - `CREATE` or `DROP` migrations are silently not even scheduled
    52  
    53  ### alter vitess_migration ... cleanup
    54  
    55  A new query is supported:
    56  
    57  ```sql
    58  alter vitess_migration '9748c3b7_7fdb_11eb_ac2c_f875a4d24e90' cleanup
    59  ```
    60  
    61  This query tells Vitess that a migration's artifacts are good to be cleaned up asap. This allows Vitess to free disk
    62  resources sooner. As a reminder, once a migration's artifacts are cleaned up, the migration is no longer revertible.
    63  
    64  ### alter vitess_migration ... complete
    65  
    66  A new query is supported:
    67  
    68  ```sql
    69  alter vitess_migration '9748c3b7_7fdb_11eb_ac2c_f875a4d24e90' complete
    70  ```
    71  
    72  This command indicates that a migration executed with `-postpone-completion` is good to complete. Behavior:
    73  
    74  - For running `ALTER`s (`online` and `gh-ost`) which are ready to cut-over: cut-over imminently (though not immediately
    75    - cut-over depends on polling interval, replication lag, etc)
    76  - For running `ALTER`s (`online` and `gh-ost`) which are only partly through the migration: they will cut-over
    77    automatically when they complete their work, as if `-postpone-completion` wasn't indicated
    78  - For queued `CREATE` and `DROP` migrations: "unblock" them from being scheduled. They'll be scheduled at the scheduler'
    79    s discretion. there is no guarantee that they will be scheduled to run immediately.
    80  
    81  ### vtctl/vtctlclient ApplySchema: ALTER VITESS_MIGRATION
    82  
    83  `vtctl ApplySchema` now supports `ALTER VITESS_MIGRATION ...` statements. Example:
    84  
    85  ```shell
    86  $ vtctl ApplySchema -skip_preflight -sql "alter vitess_migration '9748c3b7_7fdb_11eb_ac2c_f875a4d24e90' complete" commerce
    87  ```
    88  
    89  ### vtctl/vtctlclient ApplySchema: allow zero in date
    90  
    91  `vtctl/vtctlclient ApplySchema` now respects `-allow-zero-in-date` for `direct` strategy. For example, the following
    92  statement is now accepted:
    93  
    94  ```shell
    95  vtctlclient ApplySchema -skip_preflight -ddl_strategy='direct -allow-zero-in-date' -sql "create table if not exists t2(id int primary key, dt datetime default '0000-00-00 00:00:00')" commerce
    96  ```
    97  
    98  ### vtctl/vtctlclient ApplySchema -uuid_list
    99  
   100  `vtctlient ApplySchema` now support a new optional `-uuid_list` flag. It is possible for the user to explicitly specify
   101  the UUIDs for given migration(s). UUIDs must be in a specific format. If given, number of UUIDs must match the number of
   102  DDL statements. Example:
   103  
   104  ```shell
   105  vtctlclient OnlineDDL ApplySchema -sql "drop table t1, drop table t2" -uuid_list "d08f0000_51c9_11ec_9cf2_0a43f95f28a3,d08f0001_51c9_11ec_9cf2_0a43f95f28a3" commerce
   106  ```
   107  
   108  Vitess will assign each migration with given UUID in order of appearance. It is the user's responsibility to ensure
   109  given UUIDs are globally unique. If the user submits a migration with an already existing UUID, that migration never
   110  gets scheduled nor executed.
   111  
   112  ### vtctl/vtctlclient ApplySchema -migration_context
   113  
   114  `-migration_context` flag is synonymous to `-request_context`. Either will work. We will encourage use
   115  of `-migration_context` as it is more consistent with output of `SHOW VITESS_MIGRATIONS ...` which includes
   116  the `migration_context` column.
   117  
   118  ### vtctl/vtctlclient OnlineDDL ... complete
   119  
   120  Complementing the `alter vitess_migration ... complete` query, a migration can also be completed via `vtctl`
   121  or `vtctlclient`:
   122  
   123  ```shell
   124  vtctlclient OnlineDDL <keyspace> complete <uuid>
   125  ```
   126  
   127  For example:
   128  
   129  ```shell
   130  vtctlclient OnlineDDL commerce complete d08ffe6b_51c9_11ec_9cf2_0a43f95f28a3
   131  ```
   132  
   133  ### vtctl/vtctlclient OnlineDDL -json
   134  
   135  The command now accepts an optional `-json` flag. With this flag, the output is a valid JSON listing all columns and
   136  rows.
   137  
   138  ## vtadmin-web updated to node v16.13.0 (LTS)
   139  
   140  Building vtadmin-web now requires node >= v16.13.0 (LTS). Upgrade notes are given
   141  in https://github.com/vitessio/vitess/pull/9136.
   142  
   143  ### PlannedReparentShard for cluster initialization
   144  For setting up the cluster and electing a primary for the first time, `PlannedReparentShard` should be used
   145  instead of `InitShardPrimary`.
   146  
   147  `InitShardPrimary` is a forceful command and copies over the executed gtid set from the new primary to all the other replicas. So, if the user
   148  isn't careful, it can lead to some replicas not being setup correctly and lead to errors in replication and recovery later.
   149  `PlannedReparentShard` is a safer alternative and does not change the executed gtid set on the replicas forcefully. It is the preferred alternate to initialize
   150  the cluster.
   151  
   152  If using a custom `init_db.sql` that omits `SET sql_log_bin = 0`, then `InitShardPrimary` should still be used instead of `PlannedReparentShard`.
   153  
   154  ### Durability Policy flag
   155  A new flag has been added to vtctl, vtctld and vtworker binaries which allows the users to set the durability policies.
   156  
   157  If semi-sync is not being used then `-durability_policy` should be set to `none`. This is also the default option.
   158  
   159  If semi-sync is being used then `-durability_policy` should be set to `semi_sync` and `-enable_semi_sync` should be set in vttablets.
   160  
   161  ## Incompatible Changes
   162  ### Error message change when vttablet row limit exceeded:
   163  * In previous Vitess versions, if the vttablet row limit (-queryserver-config-max-result-size) was exceeded, an error like:
   164    ```shell
   165    ERROR 10001 (HY000): target: unsharded.0.master: vttablet: rpc error: code = ResourceExhausted desc = Row count exceeded 10000 (errno 10001) (sqlstate HY000) ...
   166    would be reported to the client.
   167    ```
   168    To avoid confusion, the error mapping has been changed to report the error as similar to:
   169    ```shell
   170    ERROR 10001 (HY000): target: unsharded.0.primary: vttablet: rpc error: code = Aborted desc = Row count exceeded 10000 (errno 10001) (sqlstate HY000) ...
   171    instead
   172    ```
   173  * Because of this error code change, the vttablet metric:
   174    `vttablet_errors{error_code="ABORTED"}`
   175    will be incremented upon this type of error, instead of the previous metric:
   176    `vttablet_errors{error_code="RESOURCE_EXHAUSTED"}`
   177  * In addition, the vttablet error message logged is now different.
   178    Previously, a (misleading;  due to the PoolFull) error was logged:
   179    ```shell
   180    E0112 09:48:25.420641  278125 tabletserver.go:1368] PoolFull: Row count exceeded 10000 (errno 10001) (sqlstate HY000) ...
   181    ```
   182    Post-change, a more accurate warning is logged instead:
   183    ```shell
   184    W0112 09:38:59.169264   35943 tabletserver.go:1503] Row count exceeded 10000 (errno 10001) (sqlstate HY000) ...
   185    ```
   186  * If you were using -queryserver-config-terse-errors to redact some log messages containing bind vars in 13.0-SNAPSHOT, you should now instead enable -sanitize_log_messages which sanitizes all log messages containing sensitive info
   187  
   188  ### Column types for textual queries now match MySQL's behavior
   189  
   190  The column types for certain queries performed on a `vtgate` (most notably, those that `SELECT` system variables)
   191  have been changed to match the types that would be returned if querying a MySQL instance directly: textual fields that
   192  were previously returned as `VARBINARY` will now appear as `VARCHAR`.
   193  
   194  This change should not have an impact on MySQL clients/connectors for statically typed programming languages, as these
   195  clients coerce the returned rows into whatever types the user has requested, but clients for dynamic programming
   196  languages may now start returning as "string" values that were previously returned as "bytes".
   197  
   198  ## Deprecations
   199  
   200  ### vtgate `-gateway_implementation` flag is deprecated (and ignored)
   201  
   202  Support for `discoverygateway` is being dropped. `tabletgateway` is now the only supported implementation. Scripts using
   203  this flag should be updated to remove the flag as it will be deleted in the next release.
   204  
   205  ### web/vtctld2 is deprecated and can optionally be turned off
   206  
   207  The vtctld2 web interface is no longer maintained and is planned for removal in Vitess 16. Motivation for this change and a roadmap to removing [the web/vtctld2 codebase](https://github.com/vitessio/vitess/tree/main/web/vtctld2) is given in https://github.com/vitessio/vitess/issues/9686.
   208  
   209  Vitess operators can optionally disable the vtctld2 web interface ahead of time by calling the `vtctld` process with the flag `enable_vtctld_ui=false`. For more details on this flag, see https://github.com/vitessio/vitess/pull/9614.