High Scalability Interview about Spinn3r and Tailrank Cluster Architecture

High Scalability just published an interview about the cluster architecture of Spinn3r and Tailrank.

We lift the kimono a bit and talk about plans for the future, our current hardware architecture, and our Open Source plans:

Ever feel like the blogosphere is 500 million channels with nothing on? Tailrank finds the internet’s hottest channels by indexing over 24M weblogs and feeds per hour. That’s 52TB of raw blog content (no, not sewage) a month and requires continuously processing 160Mbits of IO. How do they do that?

This is an email interview with Kevin Burton, founder and CEO of Tailrank.com. Kevin was kind enough to take the time to explain how they scale to index the entire blogosphere.

%d bloggers like this: