vitess.io/vitess@v0.16.2/changelog/10.0/10.0.0/release_notes.md (about) 1 This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components. 2 3 ## Known Issues 4 * Running binaries with `--version` or running `select @@version` from a MySQL client still shows `10.0.0-RC1` 5 * Online DDL [cannot be used](https://github.com/vitessio/vitess/pull/7873#issuecomment-822798180) if you are using the keyspace filtering feature of VTGate 6 * VReplication errors when a fixed-length binary column is used as the sharding key #8080 7 8 * A critical vulnerability [CVE-2021-44228](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228) in the Apache Log4j logging library was disclosed on Dec 9 2021. 9 The project provided release `2.15.0` with a patch that mitigates the impact of this CVE. It was quickly found that the initial patch was insufficient, and additional CVEs 10 [CVE-2021-45046](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45046) and [CVE-2021-44832](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44832) followed. 11 These have been fixed in release `2.17.1`. This release of Vitess, `v10.0.0`, uses a version of Log4j below `2.17.1`, for this reason, we encourage you to use version `v10.0.5` instead, to benefit from the vulnerability patches. 12 13 * An issue where the value of the `-force` flag is used instead of `-keep_data` flag's value in v2 vreplication workflows (#9174) is known to be present in this release. A workaround is available in the description of issue #9174. 14 15 ## Bugs Fixed 16 17 ### VTGate / MySQL compatibility 18 * Remove printing of ENFORCED word so that statements remain compatible with mysql 5.7 #7458 19 * Allow any ordering of generic options in column definitions #7459 20 * Corrects the comment handling in vitess #7581 21 * Fix regression - should be able to plan subquery on top of subquery #7682 22 * Nullable Timestamp Column Fix #7740 23 * VTGate: Fix the error messages in drop, create and alter database commands #7397 24 * VTGate: Fix information_schema query with system schema in table_schema filter #7430 25 * VTGate: Fix Set Statement in Tablet when executed with bindvars #7431 26 * VTGate: Fix for Query Serving when Toposerver is Down #7484 27 * VTGate: Add necessary bindvars when preparing queries #7493 28 * VTGate: Show anywhere plan fix to consider default keyspace #7531 29 * VTGate: Fix table parsing on VSchema generation #7511 30 * VTGate: Use the emulated MySQL version for special comments #7510 31 * VTGate: Reset Session for Reserved Connection when the connection id is not found #7539 32 * VTGate: Healthcheck: update healthy tablets correctly when a stream returns an error or timeout #7732 33 * VTGate: Fix for reserved connection usage with transaction #7646 34 * VTGate: Fix MySQL Workbench failure on login with `select current_user()` #7705 35 * VTGate: Constraint names and database names with spaces. #7745 36 * VTGate: Fix dual table query when system schema is selected database #7734 37 38 ### Other 39 * VTTablet: Correctly initialize statsTabletTypeCounts during startup #7390 40 * Backup/Restore: Respect -disable_active_reparents in backup/restore #7576 41 * Backup/Restore: check disable_active_reparents properly before waiting for position update #7703 42 43 44 ## Functionality Added or Changed 45 46 ### VTGate / MySQL compatibility / Query Serving 47 48 * VTGate: Gen4 Planner: AxB vs BxA #7274 49 * VTGAte: Gen4 fallback planning #7370 50 * VTGate: Support for CALL procedures #7287 51 * VTGate: Set default for @@workload to OLTP #7288 52 * VTGate: Change @@version and @@version_comment #7337 53 * VTGate: Fix VitessAware system variables of type boolean return NULL when MySQL is not involved #7353 54 * VTGate: Add stats for RowsAffected similar to RowsReturned #7380 55 * VTGate: Added information_schema_stats_expiry to allowed list of set vars #7435 56 * VTGate: LFU Cache Implementation #7439 57 * VTGate: Describe table to route based on table name and qualifier #7445 58 * VTGate: Olap error message fix #7448 59 * VTGate: Temporary Table support in unsharded keyspace #7411 60 * VTGate: Publish table size on schema #7444 61 * VTGate: Support for caching_sha2_password plugin in mysql/client #6716 62 * VTGate: Moving Show plan from executor to planbuilder #7475 63 * VTGate: Adds another case to merge routes for information_schema queries #7504 64 * VTGate: Add innodb_read_rows as vttablet metric #7520 65 * VTGate: Adds support for Show variables #7547 66 * VTGate: gen4: fail unsupported queries #7409 67 * VTGate: Fix Metadata in SHOW queries #7540 68 * VTGate: Update AST helper generation #7558 69 * VTGate: Avoiding addition of redundant unary operators #7579 70 * VTGate: Optimise AST rewriting #7583 71 * VTGate: Add Show Status query to vtexplain and make asthelpergen/sizegen quiet #7590 72 * VTGate: Add support for SELECT ALL #7593 73 * VTGate: Empty statement error code change in sql parsing #7618 74 * VTGate: Socket system variable to return vitess mysql socket #7637 75 * VTGate: Make DROP/CREATE DATABASE pluggable #7381 76 * VTGate: Allow Select with lock to pass through in vttablet #7584 77 * VTGate: Fix ordering in SELECT INTO and printing of strings #7655 78 * VTGate: AST Equals code generator #7672 79 * VTGate: [tabletserver] More resilient wait for schema changes #7684 80 * VTGate: Fix flush statement planner #7695 81 * VTGate: Produce query warnings for using features not supported when sharded #7538 82 * VTGate: Support for ALTER VITESS_MIGRATION statements #7663 83 * VTGate: Solve I_S queries using CNF rewriting #7677 84 * VTGate: System schema queries #7685 85 * VTGate: Make the AST visitor faster #7701 86 * VTGate: COM_PREPARE - Single TCP response packet with all MySQL Packets #7713 87 * VTGate: Replace the database name in result fields only if needed #7714 88 * VTGate: Split ast_helper into individual gen files #7727 89 * VTGate: Adds support for ordering on character fields for sharded keyspace queries #7678 90 * VTGate: Show columns query on system schema #7729 91 * VTGate: Disallow foreign key constraint on ddl #7780 92 * VTGate: VTGate: support -enable_online_ddl flag #7694 93 * VTGate: Default to false for system settings to be changed per session at the database connection level #7921 94 * VTGate: vtctl: return error on invalid ddl_strategy #7924 95 * VTGate: [10.0] Squashed backport of #7903 #7927 96 * VTGate: [10.0] Fix bug with reserved connections to stale tablets #7935 97 * VTGate: [10.0] Fix for keyspaces_to_watch regression #7936 98 * VTGate: [10.0] Update healthy tablets correctly for primary down #7937 99 * VTGate: [10.0] Allow modification of tablet unhealthy_threshold via debugEnv #7938 100 101 ### Testing 102 * Fuzzing: Add vtctl fuzzer #7605 103 * Fuzzing: Add more fuzzers #7622 104 * Fuzzing: Add 3 fuzzers for mysql endpoints #7639 105 * Fuzzing: Add oss-fuzz build script #7591 106 * Fuzzing: Add requirement for minimum length of input data #7722 107 * Fuzzing: Add new mysql fuzzer #7660 108 * Fuzzing: Add [grpcvtgateconn] fuzzer #7689 109 * Fuzzing: Make mysql fuzzer more calls during each iteration #7766 110 111 ### Performance 112 * VTGate: [perf] zero-copy tokenizer #7619 113 * VTGate: [perf: sqlparser faster formatting #7710 114 * VTGate :[perf] Cache reserved bind variables in queries #7698 115 * VTGate: [perf] sqlparser yacc codegen #7669 116 * VTGate: Making fast AST rewriter faster #7726 117 * VTGate: Cached Size Implementation #7387 118 * VTGate: Plan remove mutexes #7468 119 * LFU Cache Bug Fixes #7479 120 * [cache] Handle all possible initialization cases #7556 121 * VTGate: [servenv] provide a global flag for profiling #7496 122 * VTGate: [vttablet] Benchmarks and two small optimizations #7560 123 * [pprof]: allow stopping profiling early with a signal #7594 124 * perf: RPC Serialization #7519 125 * perf: keyword lookups in the tokenizer #7606 126 127 ### Cluster Management 128 * [vtctld] Migrate topo management RPCs #7395 129 * [vtctldclient] Set `SilenceErrors` on the root command, so we don't double-log #7404 130 * [vtctldclient] Command line flags: dashes and underscores synonyms #7406 131 * Extract the `maxReplPosSearch` struct out to `topotools` #7420 132 * Add protoutil package, refactor ISP to use it #7421 133 * Add `ErrorGroup` to package concurrency, use in `waitOnNMinusOneTablets` #7429 134 * [vtctld / wrangler] Extract some reparent methods out to functions for shared use between wrangler and VtctldServer #7434 135 * [vtctld/wrangler] Extract `EmergencyReparentShard` logic to dedicated struct and add unit tests #7464 136 * Provide named function for squashing usage errors; start using it #7451 137 * [concurrency] Add guard against potentially blocking forever in ErrorGroup.Wait() when NumGoroutines is 0 #7463 138 * Add hook for statsd integration #7417 139 * [concurrency] Add guard against potentially blocking forever in ErrorGroup.Wait() when NumGoroutines is 0 #7463 140 * Resilient rebuild keyspace graph check, tablet controls not in `RebuildKeyspaceGraph` command #7442 141 * [reparentutil / ERS] confirm at least one replica succeeded to `SetMaster`, or fail #7486 142 * [reparentutil / wrangler] Extract PlannedReparentShard logic from wrangler to PlannedReparenter struct #7501 143 * Add backup/restore duration stats #7512 144 * Refresh replicas and rdonly after MigrateServedTypes except on skipRefreshState. #7327 145 * [eparentutil] ERS should not attempt to WaitForRelayLogsToApply on primary tablets that were not running replication #7523 146 * [orchestrator] prevent XSS attack via 'orchestrator-msg' params #7526 147 * [vtctld] Add remaining reparent commands to VtctldServer #7536 148 * [reparentutil] ERS should not attempt to WaitForRelayLogsToApply on primary tablets that were not running replication #7523 149 * Shutdown vttablet gracefully if tablet record disappears #7563 150 * ApplySchema: -skip_preflight #7587 151 * Table GC: disable binary logging on best effort basis #7588 152 * Addition of waitSig pprof argument to start recording on USR1 #7616 153 * Add combine TLS certs feature #7609 154 * Check error response before attempting to access InitShardPrimary response #7658 155 * [vtctld] Migrate GetSrvKeyspace as GetSrvKeyspaces in VtctldServer #7680 156 * [vtctld] Migrate ShardReplicationPositions #7690 157 * [reparentutil | ERS] Bind status variable to each goroutine in WaitForRelayLogsToApply #7692 158 * [servenv] Fix var shadowing caused by short variable declaration #7702 159 * [vtctl|vtctldserver] List/Get Tablets timeouts #7715 160 * vtctl ApplySchema supports '-request_context' flag #7777 161 162 ### VReplication 163 164 * VReplication: vstreamer to throttle on source endpoint #7324 165 * VReplication: Throttle on target tablet #7364 166 * VReplication: Throttler: fix to client usage in vreplication and table GC #7422 167 * VReplication: MoveTables/Reshard add flags to start workflows in a stopped state and to stop workflow once copy is completed #7449 168 * VReplication: Support for caching_sha2_password plugin in mysql/client #6716 169 * VReplication: Validate SrvKeyspace during Reshard/SwitchReads #7481 170 * VReplication: [MoveTables] Refresh SrvVSchema (for Routing Rules) and source tablets (for Blacklisted Tables) on completion #7505 171 * VReplication : Data migration from another Vitess cluster #7546 172 * VReplication : [materialize] Add cells and tablet_types parameters #7562 173 * VReplication: JSON Columns: fix bug where vreplication of update statements were failing #7640 174 * VReplication: Make the frequency at which heartbeats update the _vt.vreplication table configurable #7659 175 * VReplication: Error out if binlog compression is turned on #7670 176 * VReplication: Tablet throttler: support for custom query & threshold #7541 177 * VStream API: allow aligning streams from different shards to minimize skews across the streams #7626 178 * VReplication: Backport 7809: Update rowlog for the API change made for the vstream skew alignment feature #7890 179 180 ### OnlineDDL 181 182 * OnlineDDL: update gh-ost binary to v1.1.1 #7394 183 * Online DDL via VReplication #7419 184 * Online DDL: VReplicatoin based mini stress test CI #7492 185 * OnlineDDL: Revert for VReplication based migrations #7478 186 * Online DDL: Internal support for eta_seconds #7630 187 * Online DDL: Support 'SHOW VITESS_MIGRATIONS' queries #7638 188 * Online DDL: Support for REVERT VITESS_MIGRATION statement #7656 189 * Online DDL: Declarative Online DDL #7725 190 191 ### VTAdmin 192 193 Vitess 10.0 introduces a highly-experimental multi-cluster admin API and web UI, called VTAdmin. Deploying the vtadmin-api and vtadmin-web components is completely opt-in. If you're interested in trying it out and providing early feedback, come find us in #feat-vtadmin in the Vitess slack. Note that VTAdmin relies on the new VtctldServer API, so you must be running the new grpc-vtctld service on your vtctlds in order to use it. 194 195 * VTAdmin: Add vtadmin-web build flag for configuring fetch credentials #7414 196 * VTAdmin: Add `cluster` field to vtadmin-api's /api/gates response #7425 197 * VTAdmin: Add /api/clusters endpoint to vtadmin-api #7426 198 * VTAdmin: Add /api/schemas endpoint to vtadmin-api #7432 199 * VTAdmin: [vtadmin-web] Add routes and simple tables for all entities #7440 200 * VTAdmin: [vtadmin-web] Set document.title from route components #7450 201 * VTAdmin: [vtadmin-web] Add DataTable component with URL pagination #7487 202 * VTAdmin: [vtadmin-api] Add shards to /api/keyspaces response #7453 203 * VTAdmin: [vtadmin-web] Add replaceQuery + pushQuery to useURLQuery hook #7507 204 * VTAdmin: [vtadmin-web] An initial pass for tablet filters #7515 205 * VTAdmin: [vtadmin-web] Add a Select component #7524 206 * VTAdmin: [vtadmin-api] Add /vtexplain endpoint #7528 207 * VTAdmin: [vtadmin-api] Reorganize tablet-related functions into vtadmin/cluster/cluster.go #7553 208 * VTAdmin: Three small bugfixes in Tablets table around stable sort order, display type lookup, and filtering by type #7568 209 * VTAdmin: [vtadmin] Add GetSchema endpoint #7596 210 * VTAdmin: [vtadmin/testutil] Add testutil helper to manage the complexity of recursively calling WithTestServer #7601 211 * VTAdmin: [vtadmin] Add FindSchema route #7610 212 * VTAdmin: [vtadmin-web] Add simple /schema view with table definition #7615 213 * VTAdmin: [vtadmin] vschemas api endpoints #7625 214 * VTAdmin: [vtadmin] Add support for additional service healthchecks in grpcserver #7635 215 * VTAdmin: [vtadmin] test refactors #7641 216 * VTAdmin: [vtadmin] propagate error contexts #7642 217 * VTAdmin: [vtadmin] tracing refactor #7649 218 * VTAdmin: [vtadmin] GetWorkflow(s) endpoints #7662 219 * VTAdmin: [vitessdriver|vtadmin] Support Ping in vitessdriver, use in vtadmin to healthcheck connections during Dial #7709 220 * VTAdmin: [vtadmin] Add to local example #7699 221 * VTAdmin: [vtexplain] lock #7724 222 * VTAdmin: [vtadmin] Aggregate schema sizes #7751 223 * VTAdmin: [vtadmin-web] Add comments + 'options' parameter to API hooks #7754 224 * VTAdmin: [vtadmin-web] Add common max-width to infrastructure table views #7760 225 * VTAdmin: [vtadmin-web] Add hooks + skeleton view for workflows #7762 226 * VTAdmin: [vtadmin-web] Add a hasty filter input to the /schemas view #7779 227 228 ### Other / Tools 229 230 * [rulesctl] Implements CLI tool for rule management #7712 231 232 ## Examples / Tutorials 233 234 * Source correct shell script in README #7749 235 236 ## Documentation 237 238 * Add Severity Labels document #7542 239 * Remove Google Groups references #7664 240 * Move some commas around in README.md :) #7671 241 * Add Andrew Mason to Maintainers List #7757 242 243 ## Build/CI Environment Changes 244 245 * Update java build versions to vitess 10.0.0 #7383 246 * CI: check run analysis to post JSON from file #7386 247 * Fix Dockerfiles for vtexplain and vtctlclient #7418 248 * CI: Add descriptive names to vrep shards. Update test generator script #7454 249 * CI: adding 'go mod tidy' test #7461 250 * Docker builds vitess/vtctlclient to install curl #7466 251 * Add VT_BASE_VER to vtexplain/Dockerfile #7467 252 * Enable -mysql_server_version in vttestserver, and utilize it in vttestserver container images #7474 253 * [vtctld | tests only] testtmclient refactor #7518 254 * CI: skip some tests on forked repos #7527 255 * Workflow to check make sizegen #7535 256 * Add mysqlctl docker image #7557 257 * Restore CI workflow shard 26, accidentally dropped #7569 258 * Update CODEOWNERS #7586 259 * CI: ci-workflow-gen turn string to array to reduce conflicts #7582 260 * Add percona-toolkit (for pt-osc/pt-online-schema-change) to the docker/lite images #7603 261 * CI: Use ubuntu-18.04 in tests #7614 262 * [vttestserver] Fix to work with sharded keyspaces #7617 263 * Add tools.go #7517 264 * Make vttestserver compatible with persistent data directories #7718 265 * Add vtorc binary for rpm,deb builds #7750 266 * Fixes bug that prevents creation of logs directory #7761 267 * [Java] Guava update to 31.1.1 #7764 268 * make: build vitess as static binaries by default #7795 ← Potentially breaking change 269 * make: build vitess as static binaries by default (10.0 backport) #7808 270 * java: prepare java version for release 10.0 #7922 271 272 ## Functionality Neutral Changes 273 * VTGate: Remove unused key.Destination.IsUnique() #7565 274 * VTGate: Add information_schema query on prepare statement #7746 275 * VTGate: Tests for numeric_precision and numeric_scale columns in information_schema #7763 276 * Disable flaky test until it can be fixed #7623 277 * Tests: reset stat at the beginning of test #7644 278 * Cleanup mysql server_test #7645 279 * vttablet: fix flaky tests #7543 280 * Removed unused tests for Wordpress installation #7516 281 * Fix unit test fail after merge #7550 282 * Add test with NULL input values for vindexes that did not have any. #7552 283 284 285 ## VtctldServer 286 As part of an ongoing effort to transition from the VtctlServer gRPC API to the newer VtctldServer gRPC API, we have updated the local example to use the corresponding new vtctldclient to perform InitShardPrimary (formerly, InitShardMaster) operations. 287 288 To enable the new VtctldServer in your vtctld components, update the -service_map flag to include grpc-vtctld. You may specify both grpc-vtctl,grpc-vtctld to gracefully transition. 289 290 The migration is still underway, but you may begin to transition to the new client for migrated commands. For a full listing, refer either to proto/vtctlservice.proto or consult vtctldclient --help. 291 292