Frames

Frames are the basic building blocks of the AMQP data stream.  They contain complete headers that describe queue, track and segment that is currently being constructed.  The payload of a frame (the segment being built) can be broken up into arbitrary sized byte arrays which are then reassembled based on the channel and track they are sent on.  In this way applications with memory constraints can request that frames be no bigger than what the application can fit in memory.  A typical frame header looks something like:

{
  channel: 0,
  track:1,
  is_first_frame: 1,
  is_last_frame: 1,
  is_first_segment: 1,
  is_last_segment: 1
}

Segments

Segments are like frames but instead of an arbitrary split on a data size, each segment is split on struct boundaries. That is to say, when you receive a complete segment you can be sure that it can be fully parsed. There are currently four segment types: A control segment, command segment, header segment and body segment. Command messages are currently the only type that can contain a header and body segment. For instance, the transfer command is used to send messages to and from a queue. It would contain header segments which could be used to route the entire message and it would contain a body segment which contains arbitrary data the user application cared about. Each segment is broken up into at least one frame. Multiple segments would never be sent in a single frame.

Channel and Tracks

A channel is just an integer that denotes related frames and segments. One can think of each channel being a list of incoming frames which are ordered correctly. Once the last frame in a channel is seen, the message is constructed and the channel is flushed and ready to receive a new message. In this way, multiple messages can be received at the same time by utilizing different channels but only one message can be sent on a single channel at a time.

Tracks are an exception to the one message per channel rule. There are two tracks in the current spec. The control track (track 0) and the command track (track 1). Controls preempt commands on a channel, so you can be in the middle of receiving frames on the command track and a control can come in on the control track and you must respond to that first.

This all sounds complicated but you can just think of the channel/track combination as being one entry in an hash. For instance frames coming into channel 0 and track 1 would be given the hash “0.1″:

message_channels["0.1"].add_frame(incoming)

First pass – Frame dispatching

At first it was easier to think of the frame and segment issues as different layers. At the lowest layer I would decode and dispatch each frame and then pass it off to the segment layer once a complete segment had been decoded. The segment layer would then collect the segments, relate them to each other, and then construct the full message. The frame layer looked something like this:

Flowchart showing the frame decoding layer of the kamaloka-js AMQP bindings

The dispatch would then pass it off to the segment decoder.

This became overly complicated because each segment had varying degrees of metadata and the body and header segments didn’t map to the message object very well. I could have gone ahead and created a segment object but I wanted to simplify the code.

Flattening the frame and segment layers

As it turned out flattening the model only added a couple of more steps to the current frame layer. Since frames and segments are just two different ways of breaking up a message for transfer over the wire, combining the two in the same layer made sense. What I ended up with was this:

Flowchart showing how the Kamaloka-js AMQP bindings decode frame and segments into a message

If you notice I now only create a new message if it is the first frame and first segment on a channel. When I see it is the last frame I incrementally decode the segment but I only dispatch the message once the last segment is seen. In the end, these minor adjustments allowed me to strip out a whole layer of redundant code. It also simplified the low level event code as I used to have to manage callbacks for each segment in order to construct a message. Now events only trigger once a full message is received and not when each frame or segment is received.

So why do we care about push messaging?

One can think of messaging as a conversation between two people. Poll messaging goes something like this:

Push on the other hand is a bit less chatty:

By eliminating the need to poll to know when an event has happened within Fedora’s Infrastucture, and making it easy for services to push out events in an easy fire and forget manner, we open up the door to a number of interesting possibilities. For instance, instead of implementing mail notification functionality in every service we simply create a mail notification service that listens for events and sends e-mails to people who what them. Do you want to get a notification on your desktop when your build is done instead? This makes it possible to provide that functionality without bogging down the Koji build service.

Human consumable notifications isn’t the only advantage of adding push messaging to our infrastructure. Because the data is first formatted for services to consume, notifications can be used for things like automation and synchronization. For instance someone could write a script that listens for new git checkins at fedorahosted.org and tries to package it into a private repo for personal testing.

What do we need to discuss at FUDCon?

Though this change is minimally invasive and you can ignore it if you don’t need to work with notifications, it is never the less a large undertaking. Some of the things we need to discuss are:

• Usecases – where can we benefit by adding notifications?  How do we envision the notifications will be consumed?
• Payload format – what are the pros and cons of the different ways we can encode and decode data
• Standardization – even though we have a routing protocol we still need to standardize on the type of data to expect
• Libraries – we need to make it dirt easy for infrastructure developers to add notifications to their service
• Performance and security concerns – AMQP is pretty complex so we will need to make sure all of our bases are covered when deploying the QPID servers

AMQP Sessions at FUDCon

There are a couple of talks planned and a hackfest.  Come to the talks to get a deeper understanding of AMQP and how we envision using it within Fedora and then attend the hackfest to help us map out the future of the Fedora Messaging Infrastructure.

Saturday:

• AMQP Messaging for Fedora Developers – come to my session to find out the basics of AMQP messaging and how it is relevant to Fedora’s infrastructure
• AMQP/Qpid on Fedora – The definitive guide – get a more in-depth view of AMQP in Rajith Attapattu session.  He will show you how to setup and configure the Qpid server on Fedora, use the client APIs, and where to go to find help when using AMQP.

Sunday and/or Monday:

• Get on the (Message) BUS Hackfest! – come to Jesse Keating’s hackfest to work out details and hack on the messaging infrastructure.

This along with QMF will give us the ability to send out notifications of events, monitor statistics and eventually supplement REST and XML-RPC interfaces with more efficient multi-response method calling (imagine streaming rows of data instead of waiting for the entire row set to be sent).

AMQP Handshake over Orbited

Above is a little hex dump web app I wrote to test the bindings. What you see is the start of the AMQP handshake. First I send the header indicating the version of the protocol I am expecting. I then get the ’start’ response back stating the capabilities of the broker I have connected to. I then send the ’start-ok’ packet indicating what auth mechanisms I support and the locale (I would also send a token if I were using a particular auth other than anonymous).

That is about as far as I have gotten but since the bindings are autogenerated from an XML description of the protocol most of the control and command classes are implemented. There are still some types I am trying to figure out how to support (the supported types are a magnitude above what was needed for D-Bus) and I need to smooth out the rough edges as well as write the higher level user bindings but I am off to a good start.

Code will be posted as soon as the project resources are approved and setup. Keep watching my blog for updates.

BTW: The slow response times in the time stamps are due to running inside of the firebug debugger and writing out the hex dumps on the fly using JQuery. It seems the more sophisticated JQuery and Firebug get, the more they butt heads.

TUBES the video!!!

Our latest Sugar builds now have Collabora’s excellent work on TUBES!!! I just got off an Internet video chat between myself and demo Walter Bender is doing down in Argentina. Sound was amazing. We also have a Connect 4 activity, and shared PDF activity going. Eric Blankinship is on the verge of tubafying his excellent Capture activity so kids can share photos and videos over TUBES and we have some guys tubafying a drum circle activity. I should be getting to Block Party soon and releasing a bunch of tutorials in the process.

So we belatedly proved the Senator correct. The Internet is now made up of a bunch of Tubes. There is still one thing he got wrong though. I hereby state that D-Tubes, which is D-Bus messages over Tubes, can also be called Big Truck. So now the Internet can be a Big Truck which drives over TUBES!!!!