(gs) Update: Improved Email Performance

In our recent State of the Grid post we talked about improvements we were making to the (gs) email system, specifically making it faster and more reliable. Today, we’re happy to announce that the initial improvements have been rolled out to all (gs) clusters, and email performance has been greatly improved. Read on to find out how the significant updates to the (gs) IMAP and POP3 email server is helping to keep email quick and stable on the Grid.

Introducing Dovecot

We’ve been painfully aware that many users have been unsatisfied with email stability and web response time, so we set about finding ways to make improvements. With over 600,000 mailboxes active on the Grid, even small changes had the potential to make big differences, so we looked at our email system from its foundation and evaluated the changes we could make that would bring the greatest improvements.

The first step was to make sure the email server software was as fast and reliable as it could be. We ended up switching from Courier IMAP to Dovecot.

Dovecot describes itself as “an open source IMAP and POP3 email server for Linux systems written with security primarily in mind.” It’s known for being standards compliant, high performing, and for having a small memory footprint. Our initial evaluations showed impressive performance gains, leading us to start a full roll out of the new system. We have now upgraded every cluster to Dovecot, and the load change has been impressive.

Reduced Email Load

This graph shows the average email load for all clusters just before Dovecot was rolled out on Cluster 01 while all other clusters were already updated to Dovecot. The average load for c01 hovers around the 40 “Bad” red line with clear spikes above 40, while all other clusters have much lower and more stable average loads. You’ll notice that on Clusters with Dovecot enabled email load stays below 15… a vast improvement.

System-wide Response Time Improvements

Drastically reduced load on email nodes means more reliable processing, and fewer operations on storage. This improved efficiency doesn’t just affect email… these reductions in email load means the disks can handle larger spikes in web traffic, so overall web performance has also seen improvements.

While the first cluster was upgraded on January 13th, we didn’t want to blog about it until the improvements applied to all (gs) users. We wanted to make it clear that “done” meant “done for everyone” and apologize if you felt left in the dark as the project progressed.

While Dovecot has shown to be effective in improving your email experience we know that there is still progress to be made so we’re actively working on other initiatives, including a new email architecture. Wherever possible we will continue to communicate and show the data from our next improvements.

What’s Next?

The other improvements we mentioned within the State of the Grid are in process as well. New MySQL, extra storage hardware in the form of new Solid State Drives (SSD) being added to all storage segments, and even more future email system improvements. We still have many improvements to make your experience better on the Grid and we promise that we’re not done yet.

Feel free to comment here and in the User Forums and we’ll keep you up to speed on what’s done and what’s coming.