RailSpread

====

Spread provides a high performance messaging toolkit with built in fault tolerance that serves as a unified messaging bus for distributed applications.The Spread Toolkit is language agnostic through a variety of API clients and doesn’t force a messaging format on the developer.

Rails is an MVC framework and often as an existing deployment grows ( users, data, traffic ), developers have to resort to background batch processing to keep performance in the Application tier acceptable.

Multicast

===

The Technorati Watchlist feature is powered by Spread as it allows them to have posts multicasted to groups eg. %w(politics web2.0), with each client node inspecting all posts and only cherry picking the appropriate ones.This eliminates resource intensive batch queries and throttling/scheduling such tasks to certain off peak times.

Message format

========

Opted for plain old YAML ( language agnostic ). A message is a subclass of Struct with the following properties:

  - primary_key : the primary key of the object to perform the expensive operation on
  - klass       : AR klass name
  - method      : method to invoke
  - args        : arguments to pass to the expensive method

The Message Struct is dumped to YAML before dispatch and reinstantiated by the client.

---

  !ruby/struct:RailSpread::Message
     primary_key: 1
     klass: TransportBooking
     method: :verify_availability
     args:
      - 2007-04-27 12:00:00 Z
      - 2007-05-02 12:00:00 Z

BackgrounDRB and ActiveMessaging ?

===========================

• BackgrounDRb restricts your agents / messaging clients to Ruby and doesn’t have fine grained multicast, group or point to point communication.
• ActiveMessaging adds Java, a Java Message Queue and an adapter ( Stomp ) to your existing stack.

Scaling

===========================

The very basic client included instantiates your Rails environment with a simple loop ( blocked IO ). Multiple instances can be started up on multiple hosts / virtual containers within configured Spread segments and subnets.

Dependencies

===========================

Spread 3.17.4 ( **not 4.0** ) http://www.spread.org/news/news-Spread%203.17.4.html

rb_spread 2.0 ( API compatible only with Spread 3.17.x series ) http://rbspread.sourceforge.net/

A basic Spread segment config ( spread.conf ) on localhost should be sufficient to start with.

Spread_Segment 127.0.0.255:4803 {

        localhost               127.0.0.1

}

Example

=

Start your spread daemon :

# /usr/local/sbin/spread -n localhost

Start a test client :

# spuser Join a group : j availability

Model definition :

class Booking < ActiveRecord::Base

  spreadable

  dispatches :availability,                       # generates instance method 'dispatch_availability!'
             :on => :after_create,                # callback :after_create calls dispatch_availability!
             :as => :reliable,                    # reliable delivery
             :to => ['availability'],             # to 'availability' group
             :with => :verify_availability do     # instance method we call outside Mongrel process
               [ date.date_from, date.date_to ]   # arguments passed to the above instance method
             end

  def verify_availability(date_from, date_to)
    #expensive snippet
  end

end

Fire up your client :

# ./script/spread

Launch a console :

# ./script/console

Our spuser output should look something like this :

Received REGULAR membership for group availability with 3 members, where I am member 2:

        #r3533-12#localhost
        #r8404-11#localhost
        #user#localhost

grp id is 2130706433 1179976441 3 Due to the JOIN of #r3533-12#localhost

Trigger a message from the Rails console :

>> b = Booking.find(1)

>> b.dispatch_availability!

spuser output should register the transmission :

received RELIABLE message from #r8404-11#localhost, of type 0, (endian 0) to 1 groups (161 bytes): — !ruby/struct:RailSpread::Message primary_key: 1 klass: TransportBooking method: :verify_availability args:

• 2007-04-27 12:00:00 Z
• 2007-05-02 12:00:00 Z

Our environment log:

Processing message #

  TransportBooking Load (0.000659)   SELECT * FROM bookings WHERE (bookings.`id` = 1) AND ( (bookings.`type` = 'TransportBooking' ) )

Processed message #

Great success!

Copyright © 2007 [Lourens Naude], released under the MIT license