Hypertable is out.

Hypertable is out and in the wild:

Hypertable is an open source project based on published best practices and our own experience in solving large-scale data-intensive tasks. Our goal is to bring the benefits of new levels of both performance and scale to many data-driven businesses who are currently limited by previous-generation platforms. Our goal is nothing less than that Hypertable become the world’s most massively parallel high performance database platform.

A couple of thoughts.

This already looks better than hbase.

It can run on top of KFS and HDFS. I’m not entirely happy with either of them (although KFS seems to be superior).

Once I finish up our sharded/federated database we’re running on top of MySQL I’m really tempted to look at implementing a single kernel image GFS+Bigtable clone from the ground up.

I think there’s a major advantage to implementing something in C and starting from a fresh code base.

However, I’m going to audit KFS and Hypertable as these seem like really solid starting points.

For all I know they could totally already solve my problem with just a few changes (or none at all).

MySQL is far from ideal for large databases and I think it’s going to really start showing its age in the next few years.

  1. rwmcfa1

    You might want to have a look at ThruDB, http://thrudb.org/. It’s similar in goal, written in C++ using Thrift. The backends are interchangeable. Currently thrudoc backends are disk, BerkeleyDB, partitioned MySQL, and S3 backends. With MogileFS, HBase, KFS, and HDFS all on the list of possibilities. There’s not much documentation up yet, but there’s some info about the backends at http://diststore.com/ which hasn’t been integrated yet.

%d bloggers like this: