Momentum MTA Performance Tuning Tips

Posts Tagged ‘performance’

Momentum MTA Performance Tuning Tips

Posted by Oleksiy Kovyrin under Admin-tips

This post is being constantly updated as we find out more useful information on Momentum tuning. Last update: 2012-05-05.

About 2 months ago I’ve joined LivingSocial technical operations team and one of my first tasks there was to figure out a way to make our MTAs perform better and deliver faster. We use a really great product called Momentum MTA (former Ecelerity) and it is really fast, but it is always good to be able to squeeze as much performance as possible so I’ve started looking for a ways to make our system faster.

While working on it I’ve created a set of scripts to integrate Momentum with Graphite for all kinds of crazy stats graphing, those scripts will be opensourced soon, but for now I’ve decided to share a few tips about performance-related changes we’ve made to improve our performance at least 2x:

Read the rest of this entry »

ActiveMQ + Ruby Stomp Client: How to process elements one by one

Posted by Oleksiy Kovyrin under Admin-tips, Databases, Development, My Projects

Few months ago I’ve switched one of our internal projects from doing synchronous database saves of analytics data to an asynchronous processing using starling + a pool of workers. This was the day when I really understood the power of specialized queue servers. I was using database (mostly, MySQL) for this kind of tasks for years and sometimes (especially under a highly concurrent load) it worked not so fast… Few times I worked with some queue servers, but those were either some small tasks or I didn’t have a time to really get the idea, that specialized queue servers were created just to do these tasks quickly and efficiently.

All this time (few months now) I was using starling noticed really bad thing in how it works: if workers die (really die, or lock on something for a long time, or just start lagging) and queue start growing, the thing could kill your server and you won’t be able to do something about it – it just eats all your memory and this is it. Since then I’ve started looking for a better solution for our queuing, the technology was too cool to give up. I’ve tried 5 or 6 different popular solutions and all of them sucked… They ALL had the same problem – if your queue grows, this is your problem and not queue broker’s :-/ The last solution I’ve tested was ActiveMQ and either I wasn’t able to push it to its limits or it is really so cool, but looks like it does not have this memory problem. So, we’ve started using it recently.

In this small post I’d like to describe a few things that took me pretty long to figure out in ruby Stomp client: how to make queues persistent (really!) and how to process elements one by one with clients’ acknowledgments.
Read the rest of this entry »

Advanced Squid Caching for Rails Applications: Preface

Posted by Oleksiy Kovyrin under Databases, Development, My Projects, Networks

Since the day one when I joined Scribd, I was thinking about the fact that 90+% of our traffic is going to the document view pages, which is a single action in our documents controller. I was wondering how could we improve this action responsiveness and make our users happier.

Few times I was creating a git branches and hacking this action trying to implement some sort of page-level caching to make things faster. But all the time results weren’t as good as I’d like them to be. So, branches were sitting there and waiting for a better idea.
Read the rest of this entry »

Found an Ideal I/O Scheduler for my MySQL boxes

Posted by Oleksiy Kovyrin under Admin-tips, Databases

Today I was doing some work on one of our database servers (each of them has 4 SAS disks in RAID10 on an Adaptec controller) and it required huge multi-thread I/O-bound read load. Basically it was a set of parallel full-scan reads from a 300Gb compressed innodb table (yes, we use innodb plugin). Looking at the iostat I saw pretty expected results: 90-100% disk utilization and lots of read operations per second. Then I decided to play around with linux I/O schedulers and try to increase disk subsystem throughput. Here are the results:

Read the rest of this entry »

FastSessions Rails Plugin Released

Posted by Oleksiy Kovyrin under Databases, Development, My Projects

How often do we think about our http sessions implementation? I mean, do you know, how your currently used sessions-related code will behave when sessions number in your database will grow up to millions (or, even, hundreds of millions) of records? This is one of the things we do not think about. But if you’ll think about it, you’ll notice, that 99% of your session-related operations are read-only and 99% of your sessions writes are not needed. Almost all your sessions table records have the same information: session_id and serialized empty session in the data field.

Looking at this sessions-related situation we have created really simple (and, at the same time, really useful for large Rails projects) plugin, which replaces ActiveRecord-based session store and makes sessions much more effective. Below you can find some information about implementation details and decisions we’ve made in this plugin, but if you just want to try it, then check out our project site.

Read the rest of this entry »

Top 1000 (84) MySQL Performance Tips From MySQL Camp 2006

Posted by Oleksiy Kovyrin under Development ·

Looks like MySQL Camp 2006 was really interesting and useful for its attendees and for entire MySQL community. As the result of this meeting of many MySQL-related professionals we’ve got lots of interesting publications and I want to refer one of them in this post. Very interesting list of 84 MySQL performance tips has beed created on first day of this year MySQL Camp at Google headquarters:

Index stuff.

Don’t Index Everything

Use benchmarking

Minimize traffic by fetching only what you need.

Paging/chunked data retrieval to limit

Don’t use SELECT *

Be wary of lots of small quick queries if a longer query can be more efficient

Use EXPLAIN to profile the query execution plan

Use Slow Query Log (always have it on!)

Don’t use DISTINCT when you have or could use GROUP BY

Use proper data partitions (For Cluster. Start thinking about Cluster *before* you need them)

Insert performance

Batch INSERT and REPLACE

Use LOAD DATA instead of INSERT

LIMIT m,n may not be as fast as it sounds

…

So, I think this list can be really useful for all developers/DBAs working with MySQL and want to say “Thanks” to its authors.

Looking For Optimal Solution: Ruby On Rails and Mongrel

Posted by Oleksiy Kovyrin under Development, Networks ·

This article is part of “Looking For Optimal Solution” series, devoted to testing various Ruby On Rails deployment schemes and doing some simple benchmarks on these schemes. General idea of testing is to find subset of most optimal RoR deployment schemes for different situations.

This small article is about Rails+Mongrel setup and its performance. List of other tested deployment schemes, description of testing methodology and, of course, all benchmark results you can find on “Ruby On Rails Benchmark Summary and Findings” page.

Read the rest of this entry »

High-Performance Ruby On Rails Setups Test: mongrel vs lighttpd vs nginx

Posted by Oleksiy Kovyrin under Admin-tips, Development, Networks ·

Because of not fully correct testing methodology, benchmark results are not fully correct. So, I decided to redo all tests. New benchmark results you can get in “Looking For Optimal Solution” series Summary post.

This week we have started one new project with Ruby on Rails as primary framework. My first task was to prepare runtime environment for it on one of our development servers. When I have tried to research how people doing it, I noted that there is no information about how to deploy rails application with nginx as frontend and what is performance of such solution. Before blindly make any decisions about future platform I’ve decided to make some performance tests of the some most popular rails web servers/frontends. Results of these tests you can find here with configuration samples for all software that I have used.

Read the rest of this entry »

MySQL Performance Blog – Great resource for MySQL DBAs

Posted by Oleksiy Kovyrin under Databases, Development ·

At this week I came across great blog of Peter Zaitsev and Vadim Tkachenko – two developers from High Performace Group at MySQL Inc. This blog’s name is MySQL Performance Blog and, as you probably assume, it is about misscelaneous sides of MySQL optimization and related stuff. So, if you are MySQL administrator or, even, developer, that works with MySQL, I would advise you to take a look at this blog – there are lots of useful MySQL optimization info.