More Thoughts on Constituent Replicator

Oddly enough the announcement of the constituent replicator coincides with some MySQL replication problems we’ve been feeling over the last couple of weeks.

… so here are some more thoughts and feature suggestions:

– get this in Drizzle. From what I understand they ripped out replication since the code wasn’t as elegant as the Drizzle core. I’ve heard this echoed by a number of developers so this seems like a good decision.

– Add the ability to promote a slave to a master and re-parent existing slaves to the new master – easily. I think one can do this now with log_slave_updates but it needs to be easier to setup.

– Per-statement synchronous replication support.

This could be done by a comment or an extension to the MySQL. For example:

INSERT /* replsync */ INTO FOO ...

The vast majority of my code can be asynchronous but occasionally I have to have a synchronous code block. We’ve written some code to do this ourselves but it would be nice if it were in the SQL layer.

Also, focus on getting a 1.0 release out soon. The key features we need are:

– Parallel update on slaves to increase performance
– Checksums on replication events
– Table consistency check


  1. Don’t you mean “Continuent” :)

    I hope it’s written to be fast. Our review of m/connect or whatever it was called was abysmal. They said the open source version would be Java and the commercial copy would be written in C for speed and be natively compatible with MySQL libraries. The guy reviewing it for us said neither was the case when reviewing the trial “full” copy of it…

  2. Peter

    s/constituent/continuent/g

  3. Mats Kindahl

    Well, the current replication is not ripped out of Drizzle yet. I have blueprints for a replication based on our experiences with MySQL replication, but the main motivation is to do something that is smaller and easier to both maintain and use, but with enough features to support a cloud database.

    For more information, see: http://blueprints.launchpad.net/drizzle/+spec/replication-simple

  4. Hi Kevin,

    Thanks for the excellent feedback. On the rel 1.0 features you suggest:

    1.) Parallel update. One of the design decisions we made early on was to store transactions in serial order so we could do transformations on the serial history when parallelizing. However, the link you posted on parallelizing is really thought-provoking. I think we could put something together where a single binlog gets split into independent streams that replicate out in parallel. There are some interesting failure cases and you need to make a quick up-front decision about which stream things belong to.

    2.) Checksums are in. We had them from day one.

    3.) Table consistency check. Yes, I don’t see how you can have a working system without building it in. We have an initial implementation but it’s driven by clients rather than intrinsic to the replicator. Moreover it uses Baron’s Maatkit approach which is incredibly clever but does not work across DBMS types. So we’re going to do a little refit this month to make it (a) fully intrinsic and (b) immune to DBMS types and version.

    This is already longer than your original blog post. I’m going to post on my own blog about synchronous replication and some of the other topics. We should start community frenzy to figure out how to do synchronous replication.

    Robert

    P.s., feel free to join our mailing list and speak your mind. The community is accessible at http://community.continuent.com/community/tungsten-replicator






%d bloggers like this: