In my quest for cleaner code in Kamaloka-js I have been working on simplifying the dispatch model. AMQP has some interesting features built into it to facilitate real-time functionality along with message prioritization. To accomplish this messages can be sent on different queues and tracks, and also be broken up into segments which can be further broken up into frames.
Frames
Frames are the basic building blocks of the AMQP data stream. They contain complete headers that describe queue, track and segment that is currently being constructed. The payload of a frame (the segment being built) can be broken up into arbitrary sized byte arrays which are then reassembled based on the channel and track they are sent on. In this way applications with memory constraints can request that frames be no bigger than what the application can fit in memory. A typical frame header looks something like:
{
channel: 0,
track:1,
is_first_frame: 1,
is_last_frame: 1,
is_first_segment: 1,
is_last_segment: 1
}
Segments
Segments are like frames but instead of an arbitrary split on a data size, each segment is split on struct boundaries. That is to say, when you receive a complete segment you can be sure that it can be fully parsed. There are currently four segment types: A control segment, command segment, header segment and body segment. Command messages are currently the only type that can contain a header and body segment. For instance, the transfer command is used to send messages to and from a queue. It would contain header segments which could be used to route the entire message and it would contain a body segment which contains arbitrary data the user application cared about. Each segment is broken up into at least one frame. Multiple segments would never be sent in a single frame.
Channel and Tracks
A channel is just an integer that denotes related frames and segments. One can think of each channel being a list of incoming frames which are ordered correctly. Once the last frame in a channel is seen, the message is constructed and the channel is flushed and ready to receive a new message. In this way, multiple messages can be received at the same time by utilizing different channels but only one message can be sent on a single channel at a time.
Tracks are an exception to the one message per channel rule. There are two tracks in the current spec. The control track (track 0) and the command track (track 1). Controls preempt commands on a channel, so you can be in the middle of receiving frames on the command track and a control can come in on the control track and you must respond to that first.
This all sounds complicated but you can just think of the channel/track combination as being one entry in an hash. For instance frames coming into channel 0 and track 1 would be given the hash “0.1″:
message_channels["0.1"].add_frame(incoming)
First pass – Frame dispatching
At first it was easier to think of the frame and segment issues as different layers. At the lowest layer I would decode and dispatch each frame and then pass it off to the segment layer once a complete segment had been decoded. The segment layer would then collect the segments, relate them to each other, and then construct the full message. The frame layer looked something like this:
Flowchart showing the frame decoding layer of the kamaloka-js AMQP bindings
The dispatch would then pass it off to the segment decoder.
This became overly complicated because each segment had varying degrees of metadata and the body and header segments didn’t map to the message object very well. I could have gone ahead and created a segment object but I wanted to simplify the code.
Flattening the frame and segment layers
As it turned out flattening the model only added a couple of more steps to the current frame layer. Since frames and segments are just two different ways of breaking up a message for transfer over the wire, combining the two in the same layer made sense. What I ended up with was this:
Flowchart showing how the Kamaloka-js AMQP bindings decode frame and segments into a message
If you notice I now only create a new message if it is the first frame and first segment on a channel. When I see it is the last frame I incrementally decode the segment but I only dispatch the message once the last segment is seen. In the end, these minor adjustments allowed me to strip out a whole layer of redundant code. It also simplified the low level event code as I used to have to manage callbacks for each segment in order to construct a message. Now events only trigger once a full message is received and not when each frame or segment is received.
[read this post in:
ar de es fr it ja ko pt ru zh-CN ]
