• Project Home

last modified November 25, 2008 by egj

Cabochon  

Cabochon is an event server. 




Clients send messages over HTTP to Cabochon.  Each message is marked with an event type.  Servers subscribe to event types with HTTP callbacks, and receive callbacks for event types that they're subscribed to. Cabochon itself sits in the middle and dispatches messages.  If you're confused about terminology, think of clients as "publishers" and servers as "subscribers."

When used with the CabochonClient library, Cabochon will not lose messages once the client returns from the send_message call.  It may sometimes send duplicate messages.  For most applications, this won't matter.  When it does, it should be preventable by having servers annotate data that is the result of a message with some message id, and discard messages which duplicate already-processed messages.

When a message is not accepted by a client, Cabochon tries again with exponential backoff.  After a few days, it gives up and puts the message into a failed message queue.  Messages can be returned from the failed message queue using the administative console.

Security

Cabochon presently allows anyone with the appropriate username and password to send any message.  The username and password are treated as a shared secret.  Patches to allow more granular security would be welcome.


A messaging trick


Cabochon can lose messages if they are not written to disk before the client crashes.  This could be bad.  Here's a solution:

When the client starts up, send a start-up message.  Note that this will be enqueued after any messages from before a crash.

0. If the client crashes here, it's OK because nothing has happened yet.

1. Before you begin an action, send a pre-message indicating that the action is about to begin.

2. If the client crashes here, we can't tell if the ation has been done yet.

3. Do the action

4. Likewise here.

5. Now enqueue a completed message.

6. If the client crashes here, it's OK because the message will be sent when the client restarts.

On the server side, when you get a pre-message, store it.  When you get the corresponding completed message, delete the stored pre-message.  If you get a start-up message, look for any pre-messages without corresponding completed messages.  Ask the client whether the actions in fact went through, and if they did, process them.

Get Cabochon

Cabochon can be obtained via SVN:

https://svn.openplans.org/svn/Cabochon/trunk

The client library is at:
https://svn.openplans.org/svn/CabochonClient/trunk

The beginnings of a server-subscription library are here:
https://svn.openplans.org/svn/CabochonServer/trunk

The egg for a server-subscription library are here (pypi points to this link):
https://svn.openplans.org/svn/CabochonServer/trunk#egg=cabochonserver-dev

An example server is at:
https://svn.openplans.org/svn/Statistician/trunk

RESTish API


Cabochon allows authentication via WSSE.  The username and password are configured via Cabochon's ini file.  All incoming messages must be sent with this username and password, and Cabochon sends outgoing messages with the same authentication.

Cabochon sends and recieves all values as JSON.

To create an event type:

POST to /event/
Parameters:
name: the name of the event
This returns an object with fields subscribe and fire.  The fields are URLs which, when appended to the Cabochon base, either subscribe to or fire a certain event.  See below.  There is only one subscription of a given name; this method is intended to both create and query events.

To subscribe to an event:
POST to the subscribe url
Parameters:
See http://microapps.org/HTTP_Callback

The special event named '*' will receive all messages to any queue.

To fire an event:

POST to the fire url
Parameters:
Whatever you send is passed along to subscribers
This will return the object {'status' : 'accepted'} if your request is accepted.  If it's not accepted, either correct it, or try again later.

To find a server's subscriptions:
GET /events/subscriptions
Parameters:
event_type -- the name of the event
url -- the server URL that will receive messages.

This will return an object with two fields: fire and unsubscribe.  Both are URLs.

To unsubscribe from an event:
POST to the unsubscribe URL
Parameters:
none

This will return the object {'status' : 'unsubscribed'} if you have been unsubscribed.


Python API


This is the API for the CabochonClient library:

from cabochonclient import CabochonClient
client = CabochonClient("directory to store messages", "http://url_of_cabochon_server")
#or
client = CabochonClient("directory to store messages", "http://url_of_cabochon_server", username=username, password=password)

#start up a sender in the background.  Put this wherever it is convenient, so long as it is in the same process.  Keep in
mind that messages are persistent across invocations, so don't worry about your application crashing.
sender = client.sender()
t = Thread(target=sender.send_forever)
t.setDaemon(True)
t.start()

#a convenience: set up a queue to send events
create_page_queue = client.queue("create_page")

#send a message on the queue
create_page_queue.send_message(dict(page_url="...", created_by="...", ...))

#or send a message directly
client.send_message("http://url_of_cabochon_server/events/fire_by_name/create_page", dict(...))


­

Admin console

Cabochon has an administrative console.  It's at http://host:port/admin.  Autentication is via HTTP basic auth, with username/password from a file listed in the configuration.  The console allows you to see and delete pending and failed events, and retry failed events.

Debugging

Cabochon also contains a stand-alone test client.  Run cabochon/lib/test_client.py, and subscribe localhost:8000 to '*' (or to whatever event you want to debug), and you'll see all events on the console.