What information do you want in the binlog?

The MySQL task WL#4033 Informational Events includes adding information about the original query for row-based replication. This will make it easier to debug failures and user mistakes. It also makes it easier for us to add other information to the binary log. Is there some information missing that you think we should add to the binary log? Please feel free to comment on this blog entry or send me an email at firstname.lastname@oracle.com.

MySQL 5.5 Replication Features (MySQL Sunday)

There were several requests for copies of the replication presentation at MySQL Sunday at Oracle Open World. It will be published by the conference, but for now you can download it from www.larsthalmann.com.

There were also questions about the high availability book. It is called MySQL High Availability and was written by Chuck, Mats and me.

Replication features in MySQL 5.5.3

We are happy to announce the new and exciting replication features in the MySQL 5.5.3 development release.

If you are at the MySQL User Conference, you are most welcome to join the "New Replication Features" talk at 2pm today where these will be presented in more detail.

• Semi-sync replication plugin - you can now have improved resilience of the overall replication by having a master server to wait for the slave server to collect and persist events in the relay log on a per transaction basis. As a consequence, by the time the user commits a transaction there is a guarantee that the events are already written to one or more slaves relay logs. This feature is provided as a plugin and using it is as easy as installing the plugin and enabling it.
• Slave fsync tuning - you can now fine tune relay logs and {master|relay-log}.info fsyncs operations so that corruption is much less likely on slave crashes. Three new parameters allow you to choose how often IO and SQL threads will issue file sync. You can tweak the options to your specific needs and set them on server startup.
• Relay log corruption recovery - you can now automatically recover corrupted slave relay logs. When this recovery mechanism is active, it will, on slave startup, search for corrupted relay logs and if corruption is found it discards them and contacts the master so that these are re-transferred. This feature, together with the new fsync options, increases slave resilience to crash faults.
• Replication Heartbeat - MySQL slaves will now request from the master that it sends a heartbeat (event) periodically. This is done when the slave connects to the master and the heartbeat period is configurable by the user. You can now avoid the spurious relay log rotation when the master is idle and have a more precise failure detection mechanism.
• Show Relay Logs - you can now show relay log contents, as you would do for the binlogs, by issuing 'SHOW RELAYLOG EVENTS'. This may be used for comparing the events on the master binlog against the ones in the slave relay log. In a chained replication topology, the show commands provide the ability to painlessly inspect the binary and relay logs in the servers that act both as master and slave. Additionally, it may also be used to diagnose relay log corruption.
• Per master server event filtering - you can now instruct a MySQL slave to discard events from a specific master. The 'CHANGE MASTER' has been enriched with a new 'IGNORE_SERVER_IDS = ...' clause that allows you to configure which events should be discarded. This feature is useful in circular replication topologies. In such scenarios, a server originating an event is also responsible for removing it from the chain. However, if it is removed from the chain, its events may loop forever. With this new feature, you can instruct the replacing server to remove the old server events.
• Slave Type Conversions - unlike statement-based replication, when using row-based replication replication between columns with different types was limited. Starting from 5.5, type promotion and demotion is now supported in row-based replication. For instance, one can now replicate seamlessly from a TINYINT column on the master to an INT column on the slave, from a CHAR(10) column to a VARCHAR(20) column, or even from a TEXT column to a VARCHAR(20) column.
• Individual Log Flushing - the user can now selectively flush server logs when using 'FLUSH LOGS'. Given that this command has a tight connection with binary and relay log maintenance, this means that every time the user wanted to flush such logs the other server logs would flush as well. Starting from 5.5, the user can now specify which individual log to flush by providing the log type as an argument to the 'FLUSH LOGS' command.
• Writing Non-Transactional events Directly To The Binary Log - starting from 5.5, it is safe to mix non-transactional and transactional statements in the same transaction. In row and mixed format the non-transactional statement is immediately logged instead of waiting until commit time.

To try out these features, you can download MySQL 5.5.3 from:

http://dev.mysql.com/downloads/mysql/

As always, we appreciate the community feedback. Please feel free to drop some comments or contact us.

How to contribute with Replication and Backup code

The MySQL Replication and Backup team is working with several community developers.

MySQL does a milestone release (roughly) every six months. To get patches included into a milestone release, it needs to be ready before the deadline for that milestone. All milestone releases are published, but not all will become GA-classified.

The below describes the process on how to get a contributed Replication or Backup patch included into a milestone release.

1. Get acceptance of the idea. Each accepted patch needs to be thoroughly tested (by multiple teams in different ways) and we need to ensure that we can maintain it for many years. Due to this resource requirement, we can only accept a few patches per year (this number is increasing though, which is great). When your idea is accepted, you get an assigned "coach developer" that will help your patch through the procedure to get it included into a release.
2. Get acceptance of patch. This includes multiple parts:
   
   • contributor agreement must be signed,
   • design must be accepted,
   • tests must be accepted, and
   • code must be accepted.
3. Patch gets pushed into a team repository. The Replication and Backup team push the patch into the team repository. Some more testing is done. If there are problems with the patch, it can possibly be removed from the tree.
4. Before the deadline for a milestone, the team tree gets merged into the release tree for that milestone. Release testing is done by the System QA and Server QA teams. If there are problems with the patch, it can possibly be removed from the tree.
5. The milestone release is built from the milestone repository and is published on www.mysql.com. Some milestone releases will be marked as GA releases.

Before step 2 is completed, it is hard to know for which release it can be included into.

More information about how to contribute is available here:
http://forge.mysql.com/wiki/Contributing_Code

If you have comments about this procedure or would like to contribute with Replication and Backup code, please let me know.

New Replication Features talk

In a few minutes, we will present the new replication features of 5.1 and 6.0 at the MySQL University:

Walk-through of new 5.1 and 6.0 replication features

The talk is the same as was given at the MySQL User Conference 2009.

Very welcome to join the talk!

Feature Preview: Multi-threaded Slave

We have just published Andrei's first version of the multi-threaded slave as a preview release.

Currently, the master produce a load by concurrent multiple client connections while the single slave thread execute replication events one by one. In some scenarios, this causes the slave to lag behind the master.

With the multi-threaded slave work, the replication slave will scale on multi-core machines.

This is a very early preview with serious limitations. Even so, please feel free to try it out and let us know what you think.

http://forge.mysql.com/wiki/ReplicationFeatures/ParallelSlave

MySQL 6.0.8 Feature: Interface for Semi-synchronous Replication

MySQL 6.0.8 has new interfaces for semisynchronous replication.

The interfaces make it possible to load components to ensure that all transactions are replicated to the slave before the users of the master gets acceptance of the commits.

The original code was developed by Mark Callaghan (Google) and Wei Li (Google). Zhenxing He (MySQL) ported the Google code to MySQL 6.0, extracted the functionality into pluggable components, and changed the code so that all storage engines are supported. Paul DuBois (MySQL) wrote the MySQL manual documentation.

The extraction of this patch into components is a first step in our strategy to make MySQL Replication modularized with separate loadable components for special replication functionality. With more interfaces to the server, it becomes easier to make the server behave in the way you want.

Please let us know what you think. Does it work as you expect? Do you want the synchronization to be different? Want to publish your own replication components for MySQL?

References

• MySQL manual documentation
• Plugin executables
• Plugin source code
• Original Google patches