Sharding for Faye using gem faye_shards

In this post I want to describe basic features of new gem faye_shards.

If you are not sure do you need sharding or not and why to shard - read next topics:

• High performance Publish/Subscribe solution for Ruby On Rails or Juggernaut vs. Faye
• Discussion "Stunnel for high-loaded Faye with SSL (on Node.js)"

Lets say there are 3 ways to use pub/sub:

1. Users subscribe to his own, unique chat
2. Users subscribe to global channel
3. Users subscribe to both unique and global channel.

If your application uses only 2nd option and it's high-loaded - this gem is not for you. But it perfectly fits if your app uses 1st option and it's OK to use this gem with 3rd schema. Basically sharding for 1st schema means "we are going to connect user to some fixed shard and we always know to what one".
Actually this game can be used even if you run only 1 instance of Faye - it will just provide some helper methods ;)

What kind of sharding it supports:

• It doesn't matter if you use Redis or not.
• Works with both HTTP and HTTPS. It supports cases when you run HTTP Faye with SSL Offloading (using Stunnel for example. In my previous post I've described how to configure it).
• It doesn't matter how many servers/ports you use for sharding.

Basic features it provides:

• Helpers to get URLs for users.
• Helpers to push data to faye.

Let's start with configuration file:

development:
  shards:
    -
      port: 42000
      host: 127.0.0.1
    -
      port: 42001
      host: 127.0.0.1
production:
  shards:
    -
      local_host: 10.1.1.1
      port: 42000
      host: server1.myapp.com
      secured: true
      secured_port: 32000
    -
      local_host: 10.1.1.1
      port: 42001
      host: server1.myapp.com
      secured: true
      secured_port: 32001

Required options:

• host - IP or domain name where instance of Faye is 
• running port - port, instance of Faye is listening to 

Optional params:

• local_host - local IP of server. Used for pushes from Rails instead of host if specified. Just to communicate via local interfaces. 
• secured - true/false. Specifies if there is a SSL Offloading configured for Faye instance. 
• secured_port - Required if secured is set to true. 

So, in my example for production I have 2 Faye instances listening to ports 42000 and 42001 using HTTP + Stunnel configured to listen 32000 and 32001.

Usage:
First, include module FayeShard::User::Faye to your User (or other class representing user) model.

 class User < ActiveRecord::Base
   include FayeShard::User::Faye
   ...

 end

Now you can call methods:

• user.faye_channel which returns unique channel ID for user (actually "/#{self.id}")
• user.push_to_faye(data) which pushes data to corresponding Faye shard to user's channel
• user.faye_shard - return FayeShards::Shard object, containing few nice methods.

To include Faye's client side JS use:

current_user.faye_shard.js_url(request.ssl?) # returns url for client-side JS stored on user's instance of Faye

To connect to Faye and subscribe to channel use next methods:

client = new Faye.Client(<%= raw current_user.faye_shard.url(request.ssl?).inspect %>);
client.subscribe(<%= raw current_user.faye_channel.inspect %>, function(data) {});

Methods .url and .js_url takes 1 optional boolean param - https (default is false). It specifies whether to return http URL or https. If you don't specify secured: true in faye.yml - passing true to this method will be ignored.
Note: you can pass true instead of request.ssl? if you want to connect users with http connection to https Faye.


That's it.


If you don't want / can not include module, you still can use the gem. Instead of using user.faye_channel implement you own method which returns unique token. Let's name it my_own_method_for_channel_id for next example. So, changes are:

FayeShards.shard(current_user.id).js_url(request.ssl?) # returns url for client-side JS

fayeSubscription = client.subscribe(<%= raw my_own_method_for_channel_id.inspect %>, function(data) {});

Pushing data in such case looks like:

FayeShards.shard(current_user.id).push(my_own_method_for_channel_id, data)

Pushing to global channels
Current version of gem (0.1.2) doesn't have a nice method for it. If you don't have common Redis for all your Faye instances - all you need to do is to push to all instances like that:
FayeShards.all_shards.each{ |shard| shard.push(channel, data, ext) }


In one of my next posts I'm going to write how to configure God to start/monitor Faye instances configured in faye.yml. Also, this is going to be included into gem.

PS: I've briefly explained same schema of sharding and the way to work with it in one of my previous posts. There it was an example from production project. But because of some policies I don't have permissions to show code from that application, So, that's why I've created gem faye_shards.