MDEV-37949: Implement innodb_log_archive_file_size, innodb_log_archive_path, … #4405
+1,604
−391
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
dr-m
force-pushed
the
MDEV-37949
branch
4 times, most recently
from
4a9a384 to
9d14e2c
Compare
buf_dblwr_t::create(): Create the doublewrite buffer in a single atomic mini-transaction. Do not write any log records for initializing any doublewrite buffer pages, in order to avoid recovery failure with innodb_log_archive=ON starting from the very beginning.
…target innodb_log_recovery_start: The checkpoint LSN to start recovery from. This will be useful when recovering from an archived log. innodb_log_recovery_target: The requested LSN to end recovery at. This will be useful when recovering data files that were copied as of a time that is before end of the available log.
FIXME: Implement recovery from multiple archived log files. The new setting SET GLOBAL innodb_log_archive=ON will enable log archiving as soon as the current ib_logfile0 is about to wrap around. The status variable innodb_lsn_archived will reflect the LSN since when a complete InnoDB log archive is available. Its initial value will be that of the new parameter innodb_log_archive_start. If that variable is 0 (the default), the innodb_lsn_archived will be recovered from the available log files. If innodb_log_archive=OFF, innodb_lsn_archived will be adjusted to the latest checkpoint every time a log checkpoint is executed. If innodb_log_archive=ON, the value should not change. When innodb_log_archive=ON, the setting SET GLOBAL innodb_log_file_size will affect subsequently created log files when the file that is being currently written is running out. log_t::archive_new_write(): Create and allocate a new log file, and write the outstanding data to both the current and the new file. log_t::archive_new_mmap(): Create and memory-map a new log file, and update file_size to resize_target. log_t::set_archive(my_bool): Implement SET GLOBAL innodb_log_archive. An error will be returned if non-archived SET GLOBAL innodb_log_file_size (log file resizing) is in progress. log_t::archive_set_size(): Ensure that resize_target is set for new archived log files. log_t::first_lsn: This should be the start of the current log file, to be consulted in log_t::write_checkpoint() when renaming files. log_write_buf(): Add the parameter max_length, the file wrap limit. mtr_t::finish_writer(): Specialize for innodb_log_archive=ON innodb_log_archive_update(): Assign log_sys.archived_lsn to the latest checkpoint; it should be usable. If the log buffer is about to wrap around, back off and retry. log_t::archived_mmap_switch_prepare(): Create and memory-map a new log file. log_t::append_prepare<log_t::ARCHIVED_MMAP>(): Special case. log_t::archived_mmap_switch_complete(): Switch to the buffer that was created in archived_mmap_switch_prepare(). log_t::get_archive_path(lsn_t): Return a name of an archived log file. log_t::get_next_archive_path(): Return the name of the next archived log. log_t::resize_write_low(): Adjust an assertion for log_t::ARCHIVED_MMAP. log_t::archived_mmap_switch_prepare(): Remember the file handle in log_sys.resize_log, so that write_checkpoint() will be able to invoke fchmod() on it. log_t::write_checkpoint(): Mark the old log file read-only, and update log_sys.first_lsn if we are not in the archived_mmap path. mtr_t::finish_writer(): Invoke log_close() only if innodb_log_archive=OFF.
log_t::set_archive(my_bool): Implement SET GLOBAL innodb_log_archive. The log file will be renamed. log_t::header_rewrite(my_bool): Rewrite the log file header before or after renaming the log file. The recovery of the last archived log file must tolerate also the ib_logfile0 format. TODO: Throw an error on startup if the innodb_log_archive setting does not match the log file name (and contents). TODO: Implement crash recovery for the archived-format log file. TODO: Use a different file name for encrypted archived log. log_t::get_circular_path(): Get the path name of a circular file. Replaces get_log_file_path(). log_t::get_path(): Get the name of the current log file. log_t::set_archive(): Reset first_lsn on SET innodb_log_archive=ON to be within the current file. recv_sys_t::find_checkpoint(): If the circular ib_logfile0 is missing, determine the oldest archived log file with contiguous LSN. For now, we open the first archived log file. FIXME: Validate innodb_log_archive_start, innodb_log_recovery_start, innodb_log_recovery_target FIXME: In bootstrap, write the header in the correct format. FIXME: Parse the archived log file header in the correct format. FIXME: Multi-file recovery log_t::append_archive_name(): Append the archive log file name to a path string. log_checkpoint_low(): Do not prevent a checkpoint at the start of a file. log_t::create(lsn_t): Initialize last_checkpoint_lsn.
With these changes, the following test passes: mysql-test/mtr --mysqld=--innodb-log-archive innodb.temporary_table TODO: Check the following tests, after fully implementing multi-file innodb_log_archive=ON recovery: innodb.insert_into_empty innodb.alter_kill log_t::write_checkpoint(): Write checkpoint in the archived format, and mark the previous log file (resize_log) read-only if one was open. Avoid writing a null entry when creating a log file (end_lsn==first_lsn), so that recovery will not wrongly observe an end of checkpoints. log_t::create(): In the archived format, let the header remain zero-filled. log_t::set_archive(): Fix some possible races log_t::end_lsn: The get_lsn() at the time of the latest checkpoint. log_t::log: Retain open when innodb_log_archive=ON, also for PMEM. log_t::resize_log: The new log file when innodb_log_archive=ON has switched logs but not yet completed a checkpoint in the new file. log_t::next_checkpoint_no: Widen to uint16_t. There may be up to 12288/4=3072 checkpoints in the header. recv_sys_t::find_checkpoint(): Do not read any ib_logfile* if innodb_log_archive=ON. Open non-last archived log files in read-only mode. recv_sys_t::find_checkpoint_archived(): Validate each checkpoint in the file header, and by default recover from the last one. recv_sys_t::parse(), log_parse_file(): Tweak some logic for innodb_log_archive=ON. log_t::set_recovered_checkpoint(): Set the checkpoint on recovery. Updates also the end_lsn. log_t::clear_mmap(): Clean up the logic. log_t::persist(): Even if the flushed_to_disk_lsn does not change, we may want to reset the write_lsn_offset. srv_log_rebuild_if_needed(): Never rebuild if innodb_log_archive=ON.
recv_sys_t::find_checkpoint_archived(). recv_sys_t::find_checkpoint_archived(): Fix some glitches, and enforce innodb_log_recovery_target. TODO: Multi-file recovery TODO: If there is no valid checkpoint in the last archived file, try the previous one (or start from the the first not-read-only log file that is followed by at most one not-read-only log file). TODO: Separate file name pattern for encrypted log files.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
TODO: fill description here
Release Notes
TODO: What should the release notes say about this change?
How can this PR be tested?
TODO: modify the automated test suite to verify that the PR causes MariaDB to behave as intended.
Basing the PR against the correct MariaDB version
mainbranch.This is a new feature, but for now based on the 11.4 branch so that any unrelated errors that may be found during testing can be fixed rather quickly. Merges to the
mainbranch may be blocked for weeks at a time.PR quality check