MySQL 8.0.33.0

What's new in this version:

Audit Log Notes:
- MySQL Enterprise Audit previously used tables in the mysql system database for persistent storage of filter and user account data. For enhanced flexibility, the new audit_log_database server system variable now permits specifying other databases in the global schema namespace at server startup. The mysql system database is the default setting for table storage

Compilation Notes:
- Microsoft Windows: Added MSVC Code Analysis support for Visual Studio 2017 and higher. This adds a new MSVC_CPPCHECK (defaults to OFF) CMake option that either enables or disables this analysis on the current directory and its subdirectories
- Downgraded curl deprecation warnings to -Wno-error for curl versions greater than 7.86 when MySQL is built with a GNU compiler or clang
- On macOS, added -framework CoreFoundation and -framework SystemConfiguration when linking the curl interface to link with shared system libraries as needed
- Replaced the MY_INCLUDE_SYSTEM_DIRECTORIES macro with library interfaces
- Improved CMake code to support alternative linkers
- Removed the deprecated Docs/mysql.info file from the build system
- Added a top-level .clang-tidy file and associated .clang.tidy files in the strings/ and mysys/ directories. Also enabled compdb support to enable clang-tidy usage on header files
- Removed several unmaintained or unused C++ source files for functionality such as uca-dump and uctypedump
- Added a CMake build option to enable colorized compiler output for GCC and Clang when compiling on the command line. To enable, pass -DFORCE_COLORED_OUTPUT=1 to CMake
- On Windows, also install .pdb files for associated .dll files if they are found for 3rd-party libraries
- Enterprise Linux 8 and Enterprise Linux 9 builds now use GCC 12 instead of GCC 11
- Building with -static-libgcc -static-libstdc++ now also builds the bundled protobuf with static libraries, as required.

Component Notes:
- INSTALL COMPONENT now includes the SET clause, which sets the values of component system variables while installing one or more components. The new clause reduces the inconvenience and limitations associated with the other ways of assigning variable values. For usage information, see INSTALL COMPONENT Statement

Deprecation and Removal Notes:
- User-defined collations (see Adding a Collation to a Character Set) are now deprecated. Either of the following now causes a warning to be written to the log:
- Any occurrence of COLLATE followed by the name of a user-defined collation in an SQL statement
- Use of the name of a user-defined collation as the value of collation_server, collation_database, or collation_connection
- You should expect support for user-defined collations to be removed in a future version of MySQL

MySQL Enterprise Notes:
- MySQL Enterprise Edition now provides data masking and de-identification capabilities based on components, rather than being based on a plugin library that was introduced in MySQL 8.0.13. The component implementation provides dedicated privileges to manage dictionaries and extends the list of specific types to include:
- Canada Social Insurance Number
- United Kingdom National Insurance Number
- International Bank Account Number
- Universally Unique Identifier (UUID)
- An improved table-based dictionary registry replaces the file-based dictionary used by the plugin. For a summary of the differences between the component and plugin implementations, see Data-Masking Components Versus the Data-Masking Plugin. Existing plugin users should uninstall the server-side plugin and drop its loadable functions before installing the new MySQL Enterprise Data Masking and De-Identification components

Performance Schema Notes:
- The Performance Schema Server Telemetry Traces service is added in this release. An interface which provides plugins and components a way to retrieve notifications related to SQL statements' lifetime.
- For more information on this interface, see the Server telemetry traces service section in the MySQL Source Code documentation.

The following were added:
- Status variable Telemetry_traces_supported. Whether server telemetry traces is supported. (Boolean)
- TELEMETRY_ACTIVE column was added to the threads table. Indicates whether the thread has an active telemetry session attached.

Functionality Added or Changed:
- Important Change: For platforms on which OpenSSL libraries are bundled, the linked OpenSSL library for MySQL Server has been updated to version 1.1.1t. Issues fixed in OpenSSL version 1.1.1t are described at https://www.openssl.org/news/cl111.txt
- Replication: As part of ongoing work to change old terminology used in MySQL products, the terms “master”, “slave”, and “MTS” have been replaced in error messages relating to MySQL Replication by “source”, “replica”, and “MTA”, respectively. This includes all error messages listed in messages_to_clients.txt and messages_to_error_log.txt relating to replication; the present task does not perform this replacement for messages used in other contexts.
- See the MySQL 8.0 Error Message Reference, for more information
- Replication: mysqlbinlog --start-position now accepts values up to 18446744073709551615, unless the --read-from-remote-server or --read-from-remote-source option is also used, in which case the maximum is 4294967295
- Binary packages that include curl rather than linking to the system curl library have been upgraded to use curl 7.88.1
- The use of a generated column with DEFAULT(col_name) to specify the default value for a named column is not permitted and now emits an error message

Fixed:
- NDB Cluster: Occasional temporary errors which could occur when opening a table from the NDB dictionary while repeatedly performing concurrent schema operations were not retried
- NDB Cluster: During iteration, ordered index scans retain a cursor position within each concurrently scanned ordered index fragment. Ordered index fragments are modified and balanced as a result of committing DML transactions, which can require scan cursors to be moved within the tree. When running with query threads configured (AutomaticThreadConfig set to 1), multiple threads can access the same index fragment tree structure, and the scans of multiple threads can have their cursors present in the same structure.
- The current issue arose due to an assumption in the logic for moving scan cursors when committing DML operations that all scan cursors belonged to the LDM thread owning the index fragment, which did not allow for the possibility that such fragments might belong to query threads
- InnoDB: Dead code removal
- InnoDB: Error messages related to innodb_doublewrite moved to the error log
- InnoDB: Prevent online DDL operations from accessing out-of-bounds memory
- InnoDB: ALTER TABLE ... AUTO_INCREMENT could be set to less than MAX + 1 and not forced to MAX + 1
- InnoDB: Innodb_data_pending_fsyncs could show extremely high inaccurate values because of a variable overflow
- Partitioning: Some IN() queries on partitioned tables were not always handled correctly
- Partitioning: Queries using the INDEX_MERGE optimizer hint was not handled correctly in all cases
- Replication: XA transactions whose XIDs contained null bytes could not be recovered
- Replication: When binlog_order_commits was set equal to 1, for any two transactions and for any sub-step of the commit phase, the transaction that was written to the binary log first did not always execute the sub-step first, as expected
- Replication: The binary log recovery process did not report all possible error states
- Replication: Following CHANGE REPLICATION SOURCE TO SOURCE_CONNECTION_AUTO_FAILOVER=1, failover generated a number of misleading warnings in the log that implied there were problems when in fact conditions were those expected for such a failover. These log messages have been updated accordingly
- Replication: When a transaction failed, as a side effect, extraneous error messages relating the replication data repositories were written to the log. Now in such cases, we suppress such error messages, which are not directly related to the issue of the failed transaction or its cause
- Replication: Setting binlog_order_commits to OFF could lead to a missed GTID in the next binary log file's Previous_gtids event.
- Our thanks to Yewei Xu and the Tencent team for the contribution
- Replication: Corrected the SQL statements suggested in the error message text for ER_RPL_REPLICA_ERROR_RUNNING_QUERY.
- Our thanks to Dan McCombs for the contribution
- Replication: A hash scan builds a hash of changes, scans the target table or index, and applies any matching change for the current entry. In the build phase, it uses only the before image, and skips any after image. Problems arose in some cases because generated columns were computed for the (skipped) after image, leading to replication errors. This is fixed by not computing generated columns any longer for seek-only calls such as hash scans.
- Our thanks to dc huang for the contribution
- Replication: In certain rare cases, it was possible to set gtid_mode=OFF for one session while another session, after WAIT_FOR_EXECUTED_GTID_SET() was issued by a user in this second session, was still waiting for the next GTID set from the first session. This could result in the second session waiting indefinitely for the function to return
- Group Replication: Accessing the Performance Schema replication_group_communication_information and replication_group_member_stats tables in parallel sometimes caused subsequent group replication operations to hang
- Group Replication: In certain cases, the group replication secondary node unexpectedly shut down while purging the relay log
- Group Replication: When shutting down the Group Replication plugin, the order in which the associated events were reported the error log sometimes led to confusion. To remove any doubts, we now make sure that Plugin group_replication reported: 'Plugin 'group_replication' has been stopped. is in fact the last log message relating to the shutdown, written only when all other events associated with shutting down the plugin have been logged
- Microsoft Windows: The authentication_fido_client plugin stopped responding during the authentication process if it was unable to find a FIDO device on the Windows client host
- In certain cases, CONVERT(utf8mb3_column USING UTF16) was rejected with the error Cannot convert string 'x--...' from binary to utf16
- When joining two tables on a string column, and the column from one of the tables has an additional predicate comparing it with a temporal literal, constant propagation in some cases incorrectly caused the join condition to be modified such that it used temporal rather than string semantics when comparing the strings. This caused incorrect results to be returned from the join
- Error messages returned after calling the mysql_reset_connection() C API function in a prepared statement did not identify the function name properly
- Fixed a regression in a previous fix for an issue with windowing functions.
- Our thanks to Dmitry Lenev for the contribution
- When replacing subqueries in transforms, the internal flag showing whether a given query block contains any subqueries (PROP_SUBQUERY) was not updated afterwards
- A client setting the character set to an impermissible client character set (ucs2, utf16, utf16le, or utf32) could cause unexpected behavior when the client used an authentication plugin
- EXPLAIN ANALYZE displayed 0 when the average number of rows was less than 1. To fix this, we now format numbers in the output of EXPLAIN ANALYZE and EXPLAIN FORMAT=TREE such that numbers in the range 0.001-999999.5 are printed as decimal numbers, and numbers outside this range are printed using engineering notation (for example: 1.23e+9, 934e-6). In addition, trailing zeroes are no longer printed, and numbers less than 1e-12 are printed as 0.
- This helps ensure consistent precision regardless of the number's value and improve readability, while producing minimal rounding errors
- The NTILE() function did not work correctly in all cases
- Some joins on views did not perform correctly
- Fixed an assert in sql/item_strfunc.cc that could potentially lead to issues with the SPACE() function
- Using ROW_COUNT() as the length argument to LPAD() or RPAD() did not perform as expected
- A query with a window function having an expression with a CASE function in its ORDER BY clause could lead to a server exit
- The fix for a previous issue introduced an assertion in debug builds when optimizing a HAVING clause
- When using mysqld_multi, the system that obscures "--password" usage as "--password=*****" would also match "--password-history" and "--password-require-current" definitions as "--password", but now explicitly checks for "--password=" instead
- In some cases, calling the mysql_bind_param() C API function could cause the server to become unresponsive
- The authentication_oci_client plugin was unable to open a valid configuration file if any of its entries contained an equals sign character separated by spaces (for example, key_file = /home/user/.oci/oci_api_key.pem). Now, both 'key=value' and 'key = value' entry formats are supported
- Incorrect results were returned when the result of an INTERSECT or EXCEPT operation was joined with another table. This issue affected these operations in such cases when used with either DISTINCT or ALL
- When preparing a view query, the operation used the system character set (instead of the character set stored in data dictionary) and then reported an invalid character-string error
- Prepared statements that operate on derived tables, including views, could stop unexpectedly due to problems with the code for reopening tables after an error
- Removed an assertion raised in certain cases by the RANDOM_BYTES() function in debug builds
- There was an issue in how persisted variables were set on startup, causing certain variables not to get properly set to their persisted value
- The MAKETIME() function did not perform correctly in all cases
- Some functions with multiple arguments did not produce the expected results
- A table reference in an ORDER BY outside the parenthesized query block in which the table was used, and which query block had no LIMIT or ORDER BY of its own, raised an error
- A left join with an impossible condition as part of an ON clause was not optimized as in MySQL 5.7, so that in MySQL 8.0, the query executed more quickly without the impossible condition than with it. An example of such a query, impossible condition included, is SELECT * FROM t1 JOIN t2 ON t1.c1=t2.c1 AND 1=2
- When a user defined function was part of a derived table that was merged into the outer query block, or was part of a subquery converted to a semi-join, knowledge of whether this UDF was deterministic (or not) was lost during processing
- With JSON logging enabled and an event subclass specified in the audit log filter definition, an empty item ("" : { }) was appended to the end of the logged event
- Some subqueries did not execute properly
- After the asymmetric_encrypt() component function in a SELECT query encountered a NULL field to decrypt, it could return NULL values for other non-NULL encrypted fields
- The server did not always shut down cleanly after uninstalling the audit log plugin
- Certain antijoins were not handled correctly by the server
- When the MySQL 5.7 Optimizer has 2 choices for an index to filter rows, one primary and one secondary, it picks a range scan on the secondary index because the range scan uses more key parts. MySQL 8.0 did not use this logic, instead choosing the primary index to filter rows with WHERE clause filtering. Primary key use is not suitable in such cases due to the presence of LIMIT, and due to the nature of data distribution. The secondary index was not considered while resolving order by due to constant elimination. This resulted in much different query plans in MySQL 5.7 and MySQL 8.0 for the same query.
- We solve this issue in MySQL 8.0 by skipping the constant key parts of the index during order-by evaluation only if the query is constant-optimized, which can be done at this time, but not during LIMIT analysis
- The MySQL data dictionary caches failed lookups of se_private_id values (IDs which are not found), which speeds up execution of code specific to InnoDB, relying on the fact that InnoDB does not reuse these IDs. This assumption does not necessarily hold for other storage engines, most notably NDB, where this problem was resolved previously by not using this cache.
- We extend the previous fix made for NDB so that the cache lookup is now employed only when the table uses the InnoDB storage engine
- Unexpected results were seen in some queries using DENSE_RANK(), possibly with the addition of WITH ROLLUP
- Fixed an assert raised in sql/sql_tmp_table.cc following work done previously to reimplement ROLLUP processing
- Some CTEs that did not use any tables were not always handled correctly
- Accessing rows from a window frame of a window function call present only in the query's ORDER BY list raised an error
- PERCENT_RANK() used with ORDER BY column did not return the correct result
- The --exclude-tables and --include-tables mysqlpump options did not handle views
- Changed the MySQL systemd service unit configuration from After=network-online.target to Wants=network-online.target to ensure that all configured network devices are available and have an IP address assigned before the service is started
- AVG(...) OVER (ROWS BETWEEN 1 FOLLOWING AND UNBOUNDED FOLLOWING) did not return the correct result
- A query of the form SELECT 1 FROM t1 WHERE NOT EXISTS (VALUES ROW(1), ROW(2)) caused an assert in debug builds when the subquery_to_derived optimizer switch was enabled
- mysqlimport did not escape reserved word table names when used with the --delete option
- When cloning a condition to push down to a derived table, characters in strings representing conditions were converted to utfmb4 correctly only for values less than 128 (the ASCII subset), and code points outside the ASCII subset were converted to invalid characters, causing the resulting character strings to become invalid. For derived tables without UNION, this led to problems when a column name from the derived table used characters outside the ASCII subset, and was used in the WHERE condition. For derived tables with UNION, it created problems when a character outside the ASCII subset was present in a WHERE condition.
- We fix these issues by initializing the string used for representing the condition in such cases to the connection character set
- Using --single-transaction with mysqldump version 8.0.32 required either the RELOAD or FLUSH_TABLES privilege. This requirement now applies only when both gtid_mode=ON (default OFF) and with --set-gtid-purged = ON|AUTO (default AUTO)
- Many joins using eq_ref access did not perform as well as in previous versions. This issue was first reported in MySQL 8.0.29
- Fixed a number of issues present in the internal documentation for the scramble generator algorithm in sha256_scramble_generator.cc and sha2_password_common.cc.
- Our thanks to Niklas Keller for the contribution
- CREATE USER IF NOT EXISTS added a password history entry even when the user already existed and the password was not updated. This caused a subsequent ALTER USER statement to be rejected
- A hash outer join sometimes incorrectly matched NULL with a decimal zero or an empty string that used a non-padding collation, leading to erroneous results
- An object used internally by ALTER INSTANCE RELOAD TLS was not freed until the number of readers reached 0, under the assumption is that the number of readers should reach 0 fairly frequently. The read lock held during an SSL handshake is generally an expensive operation, with network calls, so when roundtrips between the client and the server took excessively long, the lock was held for a relatively long amount of time. This meant that, when changing the value of this object and there were a sufficient number of incoming SSL connections being made, the number of readers might not reach 0 in a reasonable length of time, leaving the thread holding the lock using 100% of the CPU until the lock was released.
- We fix this by adding a wait after setting the pointer to this object to a new value, but before releasing the old object.
- Our thanks to Sinisa Milivojevic for the contribution
- If mysqldump or mysqlpump could not convert a field's default value to UTF-8 (for instance, if the field was of type BINARY and the default value did not coincide with valid UTF-8), the operation produced results that were not valid to import. Further, using the --hex-blob option did not resolve the issue. We now convert the default value to the system character set. If this fails, the server sends the value as hexadecimal instead to make it more human-readable
- A connection using the C API (libmysqlclient) client library could fail with the FUTURE crypto policy
- While cloning a temporary table for a common table expression which used shared materialization, the cloned temp table was not marked as using hash deduplication, leading to wrong results. We now set the hash field for the cloned temporary table correctly, and update the hidden field count to take this into account
- CREATE EVENT and ALTER EVENT assumed that all values passed to them (other than in a DO clause) resolved as scalars without actually checking the values. This led to assertions when any such values actually rows.
- We now perform an explicit check for the number of columns when resolving such items, and report an error when one produces a row and not a scalar value
- A view reference whose underlying field is a constant is not marked as constant when the reference is part of an inner table of an outer join. It was found that, when pushing a condition down to a derived table, the reference was stripped off and only the underlying field was cloned, which made it a constant, and led to wrong results.
- To fix this problem, we ensure that we do not push such a condition down to the derived table by adding a check to see first whether the table used by the condition matches the derived table or is a constant expression; only when it is one or the other of these do we actually push the condition down