The InnoDB log file is the transactional log, it captures all the transactions. The transaction log size is a very important factor in determining the performance of a database engine. The variable innodb_log_file_size represents the transactional log size. This log size can be changed as follows:
Set the required log file size in the MySQL configuration file.
[mysqld]
innodb_log_file_size=xG
Perform the clean shutdown. This can be verified with the help of MySQL error log. A clean shutdown is also known as slow shutdown. This process will perform a full purge of InnoDB buffer pool. The value of the innodb_fast_shutdown variable indicates the shutdown type. Slow shutdown can be performed by setting up as follows.
SET GLOBAL innodb_fast_shutdown = 0;
or
[mysqld]
innodb_fast_shutdown = 0
Take a backup of the existing InnoDB log files - just a precaution.
Remove the existing InnoDB log files. By default 2 InnoDB log files in the group, it is named as ib_logfile0, ib_logfile1.
rm /var/lib/mysql/ib_logfile*
Start the MySQL service.
sudo service mysqld start
When the server is starting, new log files are created with the same name. This can be verified by checking the error log.
tail /var/lib/mysql/mysqld.err
Note: If the existing files not removed, the server will not start and it will write the error statement on the error log.
InnoDB: Error: log file ./ib_logfile0 is of different size 0 xxxxxxxxxxx bytes
InnoDB: than specified in the .cnf file 0 xxxxxxxxxx bytes!
Why innodb_log_file_size is important?
As we have seen, the InnoDB log file is an InnoDB transaction log, to understand the importance of the InnoDB transaction log, we need to know the UNDO and REDO operations. Before that, let us understand how MySQL works when a query executes:
When a write operation is happening on MySQL, it is not directly written to the disk. The MySQL simultaneously write into InnoDB buffer pool and InnoDB transaction log. All the modifications are happening on the memory (InnoDB buffer pool). The modified pages on memory (InnoDB buffer pool) are called as Dirty pages. Once the changes are completed, the dirty pages are flushed to disk, this is called the COMMIT. So only after flushed to disk it is called committed data.
UNDO operation:
Assume that, data is being modified on the memory (InnoDB buffer pool), during this modification, if there is a ROLLBACK statement executed, the InnoDB pages should be rolled back to the original state and modified pages roll back from the ROLLBACK segment and should be removed from the InnoDB pages on buffer pool. In simple words undo operation is called as rolling back a transaction.
REDO operation:
Assume that, the modification is there in memory, the unexpected server crashed, without the transaction log, whatever there in memory gone forever. Since modified data is committed in InnoDB buffer pool(memory) but not in a disk. In order to avoid that, while restarting the server, it needs to recover the uncommitted transaction from the transaction log, The MySQL will match the sequential number of the disk and the transactional log sequence number and start to recover. In simple words, redo operation is called as replaying a committed transaction.
So what is the importance of the InnoDB log file size? Here size matters!!!.
If log file size is small: few transactions are captured in the transaction log, so it means, faster recovery but slow writes. More IOPS on a single log file.
If log file size is large: more transactions are captured in the transaction log, so it means, faster writes but slow recovery. Less IOPS on a single log file.
Finally, having huge log size will be a problem but small size also will be a problem. So what is the best size for InnoDB transaction log? From my understanding, there is no right exact value, but we have a formula, which is accepted by most of the database administrators. Here goes the rule of thumb:
InnoDB log file size: Total size of the redo logs fits in 1 hour worth of writes during peak traffic.
How much InnoDB is writing per hour?
This can be calculated by using the following statement:
mysql> pager grep seq;
mysql> show engine innodb status \G select sleep(60); show engine innodb status \G
This will give you the Log Sequence Number with one minute gap. By using this LSN we can calculate the how much writes InnoDB performs in an hour. The following sample output will give you a good idea of this:
mysql> pager grep seq;
PAGER set to 'grep seq'
mysql> show engine innodb status \G select sleep(60); show engine innodb status \G
Log sequence number 392311720379
1 row in set (0.17 sec)
1 row in set (1 min 0.00 sec)
Log sequence number 392323861206
1 row in set (0.09 sec)
mysql> select (392323861206 - 392311720379)*60/1024/1024;
+--------------------------------------------+
| (392323861206 - 392311720379)*60/1024/1024 |
+--------------------------------------------+
| 694.70369339 |
+--------------------------------------------+
1 row in set (0.00 sec)
Here, on my testing server, the InnoDB writes around 694 MB per hour. So as per my default configuration, I have 2 transaction log files, unless I change. this can be verified by running the following statement:
It means, my log file should be around 350 MB. I suggest, try this on your test environment and check the performance before implement on the production environment. Always remember to take a backup of the existing files before removing from the system. If you feel something missing or if you want to share something on the innodb_log_file_size please mention in the comment section. Please click on FOLLOW button on the right side to get more posts on database administration.
mysql> show variables like '%innodb_log_files_in_group%';
+---------------------------+-------+
| Variable_name | Value |
+---------------------------+-------+
| innodb_log_files_in_group | 2 |
+---------------------------+-------+
1 row in set (0.01 sec)
very useful interpretation! Thanks
ReplyDelete