Homo-Adminus Blog

How often do we think about our http sessions implementation? I mean, do you know, how your currently used sessions-related code will behave when sessions number in your database will grow up to millions (or, even, hundreds of millions) of records? This is one of the things we do not think about. But if you’ll think about it, you’ll notice, that 99% of your session-related operations are read-only and 99% of your sessions writes are not needed. Almost all your sessions table records have the same information: session_id and serialized empty session in the data field.

Looking at this sessions-related situation we have created really simple (and, at the same time, really useful for large Rails projects) plugin, which replaces ActiveRecord-based session store and makes sessions much more effective. Below you can find some information about implementation details and decisions we’ve made in this plugin, but if you just want to try it, then check out our project site.

FastSessions is a sessions class for ActiveRecord sessions store created to work fast (really fast). It uses some techniques which are not so widely known in developers’ community and only when they cause huge problems, performance consultants are trying to help with them.

The Problem

Original ActiveRecord sessions store is slow. It is fine for some low traffic blogs, but it is too slow to use it on some big/large/huge sites. First of all, it is slow because ActiveRecord is slow. It is powerful ORM framework, but it is overkill for such simple task as a sessions management.

That is why people created SqlSession store. It works with mysql directly with database APIs and works much faster than original AR session store. But it is still slow because:

• it creates/updates session on each hit – even dumb bots crawling your sites create thousands of thousands of useless records in your sessions table, 99% of hits do not require any session updates!
• it uses 32-char string as a key for sessions records – all databases work with string keys MUCH slower that with integers keys, so it would be much better to use integers, but we have so long session ids and all session stores use these session ids as a key.
• it uses auto_increment primary key, which causes table-level locks in InnoDB for all MySQL versions prior to 5.1.21. These table-level locks with unnecessary inserts cause really weird problems for large sites.

The Solution

FastSessions plugin was born as a hack created for Scribd.com (large RoR-based web project), which was suffering from InnoDB auto-increment table-level locks on sessions table.

So, first of all, we removed id field from the table. Next step was to make lookups faster and we’ve used a following technique: instead of using (session_id) as a lookup key, we started using (CRC32(session_id), session_id) – two-columns key which really helps MySQL to find sessions faster because almost all lookups use crc32 field only to find needed record.

And last, but most powerful change we’ve tried to make was to not create database records for empty sessions and to not save sessions data back to database if this data has not been changed during current request processing.

All of these changes were implemented and you can use them automatically after a simple plugin installation.

Controversial Decisions

Many plugin users would never think about one problem we’ve introduced when removed that auto-increment primary key, so I’d like to describe it here. The problem is following.

InnoDB groups all data in tables by primary key. This means that when we create auto-increment primary key and insert records to a table, our sessions records are grouped together and saved sequentially on the disk. But if we’ll make pretty random value (like crc32 of a random session id) a primary key, then every session record will be inserted in its own place and it will generate some random I/O which is not so good for I/O bound servers.

So, we decided to let the user choose what primary key to use in his deployment of our plugin, so if you’re going to use this module with MySQL 5.1.22+, then you’d like to set

because it will provide you with consecutive data inserts in InnoDB. Another cases when you’d like to use it is when your MySQL server is I/O bound now and you do not want to add random I/O because of randomized primary key.

Working With Old AR Sessions Table

If you do not like to loose old sessions created with default AR sessions plugin, you could set

and then all session reads will fall back to old sessions table if some session_id was not found in default fast sessions table. Old sessions table name could be set using

variable.

Installation

This plugin installation is pretty simple and described in a few steps below:

1. Install this plugin sources in your vendor/plugins directory (it could be ./script/plugin install, or piston import command – it is up to you) from our SVN reposipory. For example:

   1	$ piston import http://rails-fast-sessions.googlecode.com/svn/trunk/ vendor/plugins/fast_sessions

2. Enable ActiveRecord session store in your config/environment.rb file:

   1 2 3 4 5

   Rails::Initializer.run do |config|   ......   config.action_controller.session_store = :active_record_store   ...... end

3. Create migration for your new sessions table:

   1	$ ./script/generate fast_session_migration AddFastSessions

4. Open your newly created migration and change table_name and use_auto_increment parameters of the plugin (if you want to).
5. Run your migration:

   1	$ rake db:migrate

6. Start your application and try to perform some actions which would definitely save some data to your session. Then check your fast_sessions table (if you did not renamed it) for a records.

Downloading

Most recent version if this plugin could be found on the project’s site or in SVN repository

Author

This plugin has been created by Alexey Kovyrin. Development is sponsored by Scribd.com.