vitess.io/vitess@v0.16.2/changelog/9.0/9.0.0/release_notes.md (about) 1 This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components. 2 3 ## Incompatible Changes 4 5 The following PRs made changes to behaviors that clients might rely on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment. 6 * The update to golang 1.15 (#7204) might break systems that use TLS certificates with a common name. A fix is documented here (https://github.com/golang/go/issues/40748#issuecomment-673612108) 7 8 Vitess 9.0 is not compatible with the previous release of the Vitess Kubernetes Operator (2.2.0). A new version of the Operator (2.3.0) is available that is compatible. 9 10 ## Bugs Fixed 11 12 ### VTGate / MySQL compatibility 13 * Set Global #6957 14 * Set udv allow more expressions #6964 15 * Bug which caused the connection to not close in case of error writing an error packet #6977 16 * Bug fix #7048 for SelectNone in StreamExecute route engine #7050 17 18 ### Other 19 * Binary PK: fix bug where padding of binary columns was being done incorrectly #6963 20 * Pad non-fractional part of DECIMAL type #6967 21 * Bug fix regression in /healthz #7090 22 * Fix metadata related operation hangs when zk down #7228 23 * Fix accidentally-broken legacy vtctl output format #7285 24 * Healthcheck: use isIncluded correctly to fix replica/rdonly routing bug #6904 25 26 ## Functionality Added or Changed 27 28 ### VTGate / MySQL compatibility 29 30 * VTGate: Allow INSERT with all defaults #6969 31 * VTGate: Allow YEAR column type with length specified #6975 32 * VTGate: Retry Execute when reserved connection is lost #6983 33 * VTGate: Use ephemeral buffer when reading rows #6990 34 * VTGate: Allow time_zone in reserved connection #6998 35 * VTGate: Improved support for UNION #7007 36 * VTGate: Add DDL parser support for FULLTEXT indexes. #7001 37 * VTGate: Extend comparability of EvalResult to support hash codes #7016 38 * VTGate: Use proto equal method #7017 39 * VTGate: Fix COM_STMT_EXECUTE packet decode #7020 40 * VTGate: Adds Planning and Parsing Support for Create Index of MySQL 5.7 #7024 41 * VTGate: UNION DISTINCT support on vtgate #7029 42 * VTGate: Improve database ddl plan #7034 43 * VTGate: Support for hex & shard in vindex query #7044 44 * VTGate: Use distinct primitive to solve more queries #7047 45 * VTGate: Route using vindex for composite IN clause #7049 46 * VTGate: Optimise struct field layout #7052 47 * VTGate: Refactoring of plan building #7054 48 * VTGate: Rewrite joins written with the USING construct #6660 49 * VTGate: Add option to GetSchema to only send the row count and data length over the wire 50 #6985 51 * VTGate: Adds Planning and Parsing Support for Create Database of MySQL 5.7 #7068 52 * VTGate: Make sure to check all GROUP BY columns #7080 53 * VTGate: Separate sub-query and derived table into different structs #7081 54 * VTGate: Adds Planning and Parsing Support for Alter Database of MySQL 5.7 #7086 55 * VTGate: Convert usages of DDL struct to DDLStatement interface #7096 56 * VTGate: Adds Planning and Parsing Support for Drop Database of MySQL 5.7 #7098 57 * VTGate: Restore SHOW SCHEMAS support; fixes #7100 #7102 58 * VTGate: Refactor Code to create a separate struct for CREATE TABLE #7116 59 * VTGate: Allows for vttestserver and vtcombo to respond to VtGateExecute. #7121 60 * VTGate: Support for lock and unlock tables #7139 61 * VTGate: Merge SelectDBA routes when possible #7140 62 * VTGate: Adds support for all the rails queries using information_schema #7143 63 * VTGate: Add support for unary expression in order by column #7163 64 * VTGate: Skip query rewriting for dual table #7164 65 * VTGate: Refactor Code to create a separate struct for ALTER VSCHEMA #7173 66 * VTGate: Refactor Show plans #7185 67 * VTGate: Show privilege support #7194 68 * VTGate: Planning and Parsing Support for Drop Table, Drop View and Alter View #7178 69 * VTGate: Cache only dml and select plans #7196 70 * VTGate: Planning and Parsing Support for Alter Table #7199 71 * VTGate: Add FindAllShardsInKeyspace to vtctldserver #7201 72 * VTGate: Initial implementation of vtctld service #7128 73 * VTGate: improve-log: FAILED_PRECONDITION #7215 74 * VTGate: Default to false for system settings to be changed per session at the database connection level #7299 75 * VTGate: Route INFORMATION_SCHEMA queries #6932 76 * VTGate: Adds Planning and Parsing Support for Create Index of MySQL 5.7 #7024 77 * VTGate: Log sql which exceeds max memory rows #7055 78 * VTGate: Enable Client Session Tracking feature in mysql protocol #6783 79 * VTGate: Show columns from table_name targeted like select queries #6825 80 * VTGate: This PR adds logic to simplify subquery expressions that are simple to 81 * VTGate: Adding MySQL Check Constraints #6865 82 * VTGate: Manage read your own writes system settings #6871 83 * VTGate: Allow table_schema comparisons #6887 84 * VTGate: Additional options support for SELECT INTO and LOAD DATA #6872 85 * VTGate: Fixes vtgate which throws an error in case of empty statements #6947 86 * VTGate: [Forward Port] #6940 - Fix error handling in olap mode #6949 87 * VTGate: Adds Planning and Parsing Support for Create View of MySQL 5.7 #7060 88 * VTGate: fix error: cannot run Select on table "dual" #7118 89 * VTGate: Allow system table to be set as default database #7150 90 * VTGate: Move auto_increment from reserved to non reserved keyword #7162 91 * VTGate: Add only expr of aliasedExpr to weightstring function #7165 92 * VTGate: [9.0] don't try to compare varchars in vtgate #7271 93 * VTGate: Load Data From S3 #6823 94 * VTGate: Unnest simple subqueries #6831 95 * VTGate: Adding MySQL Check Constraints #6869 96 * VTExplain: Add sequence table support for vtexplain #7186 97 * VSchema: Support back-quoted names #7073 98 * Healthcheck: healthy list should be recomputed when a tablet is removed #7176 99 * Healthcheck: Hellcatlk wants to merge 1 commit into master from master #6953 100 101 ### Set Statement Support 102 103 Set statement support has been added in Vitess. There are [some system variables](https://github.com/vitessio/vitess/blob/main/go/vt/sysvars/sysvars.go#L147,L190) which are disabled by default and can be enabled using flag `-enable_system_settings` on VTGate. These system variables are set on the mysql server. Because they change the mysql session, using them leads to the Vitess connection no longer using the connection pool and forcing dedicated connections. 104 105 106 ### VReplication 107 108 * VReplication: refactored and enhanced support for JSON columns #6829 109 * VReplication: Don't update tx timestamp on heartbeat #6930 110 * VReplication E2E Tests: Refactored tests for readability and attempting to fix flakiness #6991 111 * VRepl/Tablet Picker: improve observability of selected tablet #6999 112 * VReplication: Handle comment statement type in vstreamer #7092 113 * VReplication e2e: Fine tuned test to reduce flakiness and added more logging to debug future flakiness #7138 114 * VReplication: Make relay log size & rows configurable. #6992 115 * VReplication: New workflows cli UX. Allow reads/writes to be switched independently #7071 116 * VReplication: DropSources: change table rename logic #7230 117 * VReplication: MoveTables: delete routing rules and update vschema on Complete and Abort #7234 118 * VReplication: V2 Workflow Start: wait for streams to start and report errors if any while starting a workflow #7248 119 * VReplication: Ignore temp tables created by onlineddl #7159 120 * VReplication: Set time zone to UTC while streaming rows #6845 121 * VReplication: Materialization and character sets: Add test to verify/demo a workaround for charset issues while using string functions in filters #6847 122 * VReplication: Tool to diagnose vreplication issues in production #6892 123 * VReplication: Allow multiple blacklists for master #6816 124 * VStreamer Field Event: add allowed values for set/enum #6981 125 * VDiff: lock keyspace while snapshoting, restart target in case of errors #7012 126 * VDiff: make enums comparable #6880 127 * VDiff: add ability to limit number of rows to compare #6890 128 * VDiff/Tablet Picker: fix issue where vdiff sleeps occasionally for tablet picker retry interval #6944 129 * [vtctld]: fix error state in Workflow Show #6970 130 * [vtctld] Workflow command: minor fixes #7008 131 * MoveTables: validate that source tables exist, move all tables #7018 132 * SwitchWrites bug: reverse replication workflows can have wrong start positions #7169 133 134 ### VTTablet 135 136 * VTTablet: fast and reliable state transitions #7011 137 * VTTablet: don't shutdown on too many connections #7039 138 * VTTablet: debug/env page to change variables in real-time #7189 139 * VTTablet: Adds better errors when there are timeouts in resource pools #7002 140 * VTTablet: Return to re-using server IDs for binlog connections #6941 141 * VTTablet: Correctly initialize the TabletType stats #6989 142 * Backup: Use pargzip instead of pgzip for compression. #7037 143 * Backup: Add s3 server-side encryption and decryption with customer provided key #7088 144 145 ### OnlineDDL 146 147 * Online DDL: follow ups in multiple trajectories #6901 148 * Online DDL: cancel running migrations executed by another tablet #7006 149 * OnlineDDL: Adding `ddl_strategy` session variable #7042 150 * Online DDL: ddl_strategy session variable and vtctl command line argument #7045 151 * Online DDL: Removing online ddl query hint from ALTER TABLE #7069 152 * Online DDL: vtgate -ddl-strategy flag renamed to -ddl_strategy #7074 153 Automatically retry migration that was interrupted during master failover 154 Automatically terminate migrations run by a failed tablet 155 * Online DDL:request_context/migration_context #7082 156 * Online DDL: Support CREATE, DROP statements in ApplySchema and online DDL #7083 157 * Online DDL: ddl_type column #7097 158 * OnlineDDL: "cancel-all" command to cancel all pending migrations in keyspace #7099 159 * OnlineDDL: Support `vtctl OnlineDDL <keyspace> show <context>` #7145 160 * OnlineDDL: Normalizing Online-DDL queries #7153 161 * Online DDL: ddl_strategy=direct #7172 162 * Online DDL: Executor database pool size increase #7206 163 * Online DDL: DROP TABLE translated to RENAME TABLE statement #7221 164 * Online DDL: Adding @@session_uuid to vtgate; used as 'context' #7263 165 * Online DDL: ignore errors if extracted gh-ost binary is identical to installed binary #6928 166 * Online DDL: Table lifecycle: skip time hint for unspecified states #7151 167 * Online DDL: Migration uses low priority throttling #6830 168 * Online DDL: Fix parsing of online-ddl command line options #6900 169 * OnlineDDL bugfix: make sure schema is applied on tablet #6910 170 * OnlineDDL: request_context/migration_context #7082 171 * OnlineDDL: Fix missed rename in onlineddl_test #7148 172 * OnlineDDL: Online DDL endtoend tests to support MacOS #7168 173 174 ### VTadmin 175 176 * VTadmin: Initial vtadmin-api, clusters, and service discovery #7187 177 * VTadmin: The tiniest possible first implementation of vtadmin-web #7218 178 * VTadmin: Add cluster protos to discovery and vtsql package constructors #7224 179 * VTadmin: Add static file service discovery implementation #7229 180 * VTadmin: Query vtadmin-api from vtadmin-web with fetch + react-query #7239 181 * VTadmin: Move allow_alias option in MySqlFlag enum to precede the aliased IDs #7166 182 * [vtctld]: vtctldclient generator #7238 183 184 ### Other 185 186 * Fix comment typo #6974 187 * Fix all occurrences of `fmt.Sprint(x)` where x is `int` #7244 188 * Fix incorrect comments #7257 189 * Fix comment for IDPool #7212 190 * IsInternalOperationTableName: see if a table is used internally by vitess #7104 191 * Add timeout for mysqld_shutdown #6849 192 * Should receive healthcheck updates from all tablets in cells_to_watch #6852 193 * Workflow listall with no workflows was missing newline #6853 194 * Allow incomplete SNAPSHOT keyspaces #6863 195 196 ## Examples / Tutorials 197 198 * Examples/operator: fix tags and add vtorc example #7358 199 * local docker: copy examples/common into /vt/common to match MoveTables user guide #7252 200 * Update docker-compose examples to take advantage of improvements in Vitess #7009 201 202 ## Documentation 203 204 * Vitess Slack Guidelines v1.0 #6961 205 * Do vschema_customer_sharded.json before create_customer_sharded.sql #7210 206 * Added readme for the demo example #7226 207 * Adding @shlomi-noach to CODEOWNERS #6855 208 * Add Rohit Nayak to maintainers #6903 209 * 7.0.3 Release Notes #6902 210 * 8_0_0 Release Notes #6958 211 * Update maintainers of Vitess #7093 212 * Updating Email Address #7095 213 * Update morgo changes #7105 214 * Move PR template to .github directory #7126 215 * Fix trivial typo #7179 216 * Add @ajm188 + @doeg to CODEOWNERS for vtctld service files #7202 217 * Add @ajm188 + @doeg as vtadmin codeowners #7223:w 218 219 220 ## Build Environment Changes 221 222 * Clean up plan building test cases #7057 223 * Fix unit test error #6953, #6993 224 * Fixing the 5.6 builds of vitess/lite #6960 225 * Pin mariadb to use mariadb-server-10.2 #6966 226 * Replace vitess:base with vitess:lite images for docker-compose services #7004 227 * Fix flakey TestParallelRunnerApprovalFirstRunningSecondRunning test #7014 228 * Allow custom image tags in compose #7043 229 * Support statsd for vitess #7072 230 * Add vtctl to make install-local #7125 231 * Updating Java unit tests for JDK9+ compatibility #7144 232 * Add Go Version to Bootstrap Image #7182 233 * Update Vitess v8.0 images #7174 234 * Fix broken package ref in UBI docker build #7183 235 * Convert CentOS extra packages installation to yum instead of downloading #7188 236 * Make docker_local: fix missing mysql_server package #7213 237 * Add unit test case to improve test coverage for go/sqltypes/result.go #7227 238 * Update Golang to 1.15 #7204 239 * Add linter configuration #7247 240 * Modify targets to restore behavior of make install #6842 241 * Download zookeeper 3.4.14 from archive site #6865 242 * Bump junit from 4.12 to 4.13.1 in /java #6870 243 * Fix ListBackups for gcp and az to work with root directory #6873 244 * Pulling bootstrap resources from vitess-resources #6875 245 * [Java] Bump SNAPSHOT version to 9.0 after Vitess release 8.0 #6907 246 * Change dependencies for lite builds #6933 247 * Truncate logged query in dbconn.go. #6959 248 * [GO] go mod tidy #7137 249 * goimport proto files correctly #7264 250 * Cherry pick version of #7233 for release-9.0 #7265 251 * Update Java version to 9.0 #7369 252 * Adding curl as dependency #6965 253 254 ## Functionality Neutral Changes 255 256 * Healthcheck: add unit test for multi-cell replica configurations #6978 257 * Healthcheck: Correct Health Check for Non-Serving Types #6908 258 * Adds timeout to checking for tablets. #7106 259 * Remove deprecated vtctl commands, flags and vttablet rpcs #7115 260 * Fixes comment to mention the existence of reference tables. #7122 261 * Updated pull request template to add more clarity #7193 262 * Redact password #7198 263 * action_repository: no need for http.Request #7124 264 * Testing version upgrade/downgrade path from/to 8.0 #7323 265 * Use `context` from Go's standard library #7235 266 * Update `operator.yaml` backup engine description #6832 267 * Docker - upgrade to Debian Buster #6833 268 * Updating azblob to remove directory after removing backup #6836 269 * Fixing some flaky tests #6874 270 * Flaky test: attempt to fix TestConnection in go/test/endtoend/messaging #6879 271 * Stabilize test #6882 272 * Tablet streaming health fix: never silently skip health state changes #6885 273 * Add owners to /go/mysql #6886 274 * Fixes a bug in Load From statement #6911 275 * Query consolidator: fix to ignore leading margin comments #6917 276 * Updates to Contacts section as Reporting #7023 277 * Create pull_request_template #7027 278 * Fixed pull request template path #7062 279 280 ## Backport 281 * Backport: [vtctld] Fix accidentally-broken legacy vtctl output format #7292 282 * Backport #7276: Vreplication V2 Workflows: rename Abort to Cancel #7339 283 * Backport #7297: VStreamer Events: remove preceding zeroes from decimals in Row Events 284 * Backport #7255: VReplication DryRun: Report current dry run results for v2 commands #7345 285 * Backport #7275: VReplication: Miscellaneous improvements #7349 286 * Backport 7342: Workflow Show: use timeUpdated to calculate vreplication lag #7354 287 * Backport 7361: vtctl: Add missing err checks for VReplication v2 #7363 288 * Backport 7297: VStreamer Events: remove preceding zeroes from decimals in Row Events #7340