Back in November 2009 I was working on a project to port Scribd.com code base to Rails 2.2 and noticed that some old plugins we were using in 2.1 were abandoned by their authors. Some of them were just removed from the code base, but one needed a replacement – that was an old plugin called acts_as_readonlyable that helped us to distribute our queries among a cluster of MySQL slaves. There were some alternatives but we didn’t like them for one or another reasons so we’ve decided to go with creating our own ActiveRecord plugin, that would help us scale our databases out. That’s the story behind the first release of DbCharmer.
Today, six months after the first release of the gem and we’ve moved it to gemcutter (which is now the official gems hosting) and we’re already at version 1.6.11. The gem was downloaded more than 2000 times. There are (at least) 10+ large users that rely on this gem to scale their products out. And (this is the most exciting) we’ve added tons of new features to the product.
Here are the main features added since the first release:
- Much better multi-database migrations support including default migrations connection changing.
- We’ve added ActiveRecord associations preload support that makes it possible to move eager loading queries to the same connection where your finder queries go to.
- We’ve improved ActiveRecord’s query logging feature and now you can see what connections your queries executed on (and yes, all those improvements are colorized
).
- We’ve added an ability to temporary remap any ActiveRecord connections to any other connections for a block of code (really useful when you need to make sure all your queries would go to some non-default slave and you do not want to mess with all your models).
- The most interesting change: we’ve implemented some basic sharding functionality in ActiveRecord which currently is being used in production in our application.
As you can see now DbCharmer helps you to do three major scalability tasks in your Rails projects:
- Master-Slave clusters to scale out your Rails models reads.
- Vertical sharding by moving some of your models to a separate (maybe even dedicated) servers and still keep using AR associations
- Horizontal sharding by slicing your models data to pieces and placing those pieces into different databases and/or servers.
So, If you didn’t check DbCharmer out yet and you’re working on some large rails project that is (or going to be) facing scalability problems, go read the docs, download/install the gem and prove them that Rails CAN scale!
loops is a small and lightweight framework for Ruby on Rails and Merb created to support simple background loops in your application which are usually used to do some background data processing on your servers (queue workers, batch tasks processors, etc).
Originally loops plugin was created to make our (Scribd.com) own loops code more organized. We used to have tens of different modules with methods that were called with script/runner and then used with nohup and other not so convenient backgrounding techniques. When you have such a number of loops/workers to run in background it becomes a nightmare to manage them on a regular basis (restarts, code upgrades, status/health checking, etc).
After a short time of writing our loops in more organized ways we were able to generalize most of the loops code so now our loops look like a classes with a single mandatory public method called run. Everything else (spawning many workers, managing them, logging, backgrounding, pid-files management, etc) is handled by the plugin itself.
The major idea behind this small project was to create a deadly simple and yet robust framework to be able to run some tasks in background and do not think about spawning many workers, restarting them when they die, etc. So, if you need to be able to run either one or many copies of your worker or you do not want to think about re-spawning dead workers and do not want to spend megabytes of RAM on separate copies of Ruby interpreter (when you run each copy of your loop as a separate process controlled by monit/god/etc), then I’d recommend you to try this framework — you’ll like it.
For more information, visit the project site and, of course, read the sources
Few days ago we were chatting in our corporate Campfire room and one of the guys asked me what do I think about Rails developers hiring process, what questions I’d ask a candidate, etc… This question started really long and interesting discussion and I’d like to share my thoughts on this question in this post.
Read the rest of this entry »
Today I’ve managed to finish initial version of our bounces-handler package we use for mailing-related stuff in Scribd.
Bounces-handler package is a simple set of scripts to automatically process email bounces and ISP‘s feedback loops emails, maintain your mailing blacklists and a Rails plugin to use those blacklists in your RoR applications.
This piece of software has been developed as a part of more global work on mailing quality improvement in Scribd.com, but it was one of the most critical steps after setting up reverse DNS records, DKIM and SPF.
The package itself consists of two parts:
- Perl scripts to process incoming email:
- bounces processor — could be assigned to process all your bounce emails
- feedback loops messages processor — more specific for Scribd, but still – could be modified for your needs (will be released soon).
- Rails plugin to work with mailing blacklists
For more information, please check our README file. If you have any questions, comments or suggestions, please leave them here as a comments and I’ll try to reply as soon as possible.
Hello my dear readers.
Today I have a question for all of you. What platforms (32bit or 64 bit) do you use for your servers with more than 4Gb RAM? I’m asking because recently we‘ve hit few really weird bugs in Linux kernels 2.6.18 to 2.6.22 and all those bugs were PAE-related. Now I’d really love to move all machines to 64-bit, but I’m in doubt because we don’t know too much about Rails stack (ruby, mongrel, haproxy) on 64-bit platforms (all our DB boxes are 64-bit of course).
So, please drop me a line if you have any experience (negative or positive) with Rails platform on 64-bit machines. I’d really appreciate your help.
Last few days one of our customers (one of the largest Ruby on Rails sites on the Net) was struggling to solve some really strange problem – once upon a time they were getting an error from ActiveRecord on their site:
1
| (ActiveRecord::StatementInvalid) "Mysql::Error: Lock wait timeout exceeded; try restarting transaction: UPDATE some_table..... |
They have innodb_lock_wait_timeout set to 20 seconds. After a few hours of looking for strange transactions we were decided to create s script to dump SHOW INNODB STATUS and SHOW FULL PROCESSLIST commands output to a file every 10 seconds to catch one of those moments when this error occurred.
Today we’ve got next error and started digging in our logs…
Read the rest of this entry »
This article is part of “Looking For Optimal Solution” series, devoted to testing various Ruby On Rails deployment schemes and doing some simple benchmarks on these schemes. General idea of testing is to find subset of most optimal RoR deployment schemes for different situations.
This small article is about Rails+Mongrel setup and its performance. List of other tested deployment schemes, description of testing methodology and, of course, all benchmark results you can find on “Ruby On Rails Benchmark Summary and Findings” page.
Read the rest of this entry »
