Building a Scalable and Modern Infrastructure at CARFAX

The CARFAX vehicle history database using MongoDB contains over 14 billion documents in a twelve shard cluster that replicates to multiple data centers.

This presentation will be a step by step walk through of how we deploy our servers, manage high volume reads and writes, and our configuration for high availability. By automating everything from the operating system install up we are able deploy complete replica clusters quickly and efficiently. Using distributed processing and message queuing through RabbitMQ we load millions of new documents each day with a projected growth over 1.5 billion per year. Through the use of tagging, server configuration, and read settings we deliver content with high consistency and availability.

About Jai Hirsch

Jai is a Senior Systems Architect for Data Technologies at CARFAX and a member of the MongoDB Masters group. He started writing software in 1995 and has been writing JVM-based data transformations and batch services since 2006. The past few years he has been focused on high volume MongoDB implementations using distributed processing through RabbitMQ. Jai is one of the lead engineers to convert the CARFAX vehicle history database from a legacy distributed key value store on OpenVMS to a now 14+ billion record, 10.5 terabyte MongoDB instance that is replicated to multiple data-centers and grows over 1.5 billion documents per year.

More About Jai »