Navigation

  • index
  • next |
  • previous |
  • Qpid Dispatch 0.6.1 documentation »
  • Qpid Dispatch Router Book »
  • 2. Using Qpid Dispatch »

Table Of Contents

  • 2.5. Indirect Waypoints and Auto-Links
    • 2.5.1. Queue Waypoint Example
    • 2.5.2. Sharded Queue Example
    • 2.5.3. Dynamically Adding Shards

Previous topic

2.4. Link Routing

Next topic

3. Technical Details and Specifications

Quick search

2.5. Indirect Waypoints and Auto-Links¶

This feature was introduced in Qpid Dispatch 0.6. It is a significant improvement on an earlier somewhat experimental feature called Waypoints.

Auto-link is a feature of Qpid Dispatch Router that enables a router to actively attach a link to a node on an external AMQP container. The obvious application for this feature is to route messages through a queue on a broker, but other applications are possible as well.

An auto-link manages the lifecycle of one AMQP link. If messages are to be routed to and from a queue on a broker, then two auto-links are needed: one for sending messages to the queue and another for receiving messages from the queue. The container to which an auto-link attempts to attach may be identified in one of two ways:

  • The name of the connector/listener that resulted in the connection of the container, or
  • The AMQP container-id of the remote container.

2.5.1. Queue Waypoint Example¶

Here is an example configuration for routing messages deliveries through a pair of queues on a broker:

connector {
    name: broker
    role: route-container
    host: <hostname>
    port: <port>
    sasl-mechanisms: ANONYMOUS
}

address {
    prefix: queue
    waypoint: yes
}

autoLink {
    addr: queue.first
    dir: in
    connection: broker
}

autoLink {
    addr: queue.first
    dir: out
    connection: broker
}

autoLink {
    addr: queue.second
    dir: in
    connection: broker
}

autoLink {
    addr: queue.second
    dir: out
    connection: broker
}

The address entity identifies a namespace (queue.*) that will be used for routing messages through queues via autolinks. The four autoLink entities identify the head and tail of two queues on the broker that will be connected via auto-links.

If there is no broker connected, the auto-links shall remain inactive. This can be observed by using the qdstat tool:

$ qdstat --autolinks
AutoLinks
  addr          dir  phase  link  status    lastErr
  ===================================================
  queue.first   in   1            inactive
  queue.first   out  0            inactive
  queue.second  in   1            inactive
  queue.second  out  0            inactive

If a broker comes online with a queue called queue.first, the auto-links will attempt to activate:

$ qdstat --autolinks
AutoLinks
  addr          dir  phase  link  status  lastErr
  ======================================================================
  queue.first   in   1      6     active
  queue.first   out  0      7     active
  queue.second  in   1            failed  Node not found: queue.second
  queue.second  out  0            failed  Node not found: queue.second

Note that two of the auto-links are in failed state because the queue does not exist on the broker.

If we now use the Qpid Proton example application simple_send to send three messages to queue.first via the router:

$ python simple_send.py -a 127.0.0.1/queue.first -m3
all messages confirmed

and then look at the address statistics on the router:

$ qdstat -a
Router Addresses
  class   addr           phs  distrib   in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  ========================================================================================================
  mobile  queue.first    1    balanced  0        0      0       0      0   0    0     0        0
  mobile  queue.first    0    balanced  0        1      0       0      3   3    0     0        0

we see that queue.first appears twice in the list of addresses. The phs, or phase column shows that there are two phases for the address. Phase ‘0’ is for routing message deliveries from producers to the tail of the queue (the out auto-link associated with the queue). Phase ‘1’ is for routing deliveries from the head of the queue to subscribed consumers.

Note that three deliveries have been counted in the “in” and “out” columns for phase ‘0’. The “in” column represents the three messages that arrived from simple_send and the “out” column represents the three deliveries to the queue on the broker.

If we now use simple_recv to receive three messages from this address:

$ python simple_recv_noignore.py -a 127.0.0.1:5672/queue.first -m3
{u'sequence': int32(1)}
{u'sequence': int32(2)}
{u'sequence': int32(3)}

We receive the three queued messages. Looking at the addresses again, we see that phase ‘1’ was used to deliver those messages from the queue to the consumer.

$ qdstat -a
Router Addresses
  class   addr           phs  distrib   in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
  ========================================================================================================
  mobile  queue.first    1    balanced  0        0      0       0      3   3    0     0        0
  mobile  queue.first    0    balanced  0        1      0       0      3   3    0     0        0

Note that even in a multi-router network, and with multiple producers and consumers for queue.first, all deliveries will be routed through the queue on the connected broker.

2.5.2. Sharded Queue Example¶

Here is an extension of the above example to illustrate how Qpid Dispatch Router can be used to create a distributed queue in which multiple brokers share the message-queueing load.

connector {
    name: broker1
    role: route-container
    host: <hostname>
    port: <port>
    sasl-mechanisms: ANONYMOUS
}

connector {
    name: broker2
    role: route-container
    host: <hostname>
    port: <port>
    sasl-mechanisms: ANONYMOUS
}

address {
    prefix: queue
    waypoint: yes
}

autoLink {
    addr: queue.first
    dir: in
    connection: broker1
}

autoLink {
    addr: queue.first
    dir: out
    connection: broker1
}

autoLink {
    addr: queue.first
    dir: in
    connection: broker2
}

autoLink {
    addr: queue.first
    dir: out
    connection: broker2
}

In the above configuration, there are two instances of queue.first on brokers 1 and 2. Message traffic from producers to address queue.first shall be balanced between the two instance and messages from the queues shall be balanced across the collection of subscribers to the same address.

2.5.3. Dynamically Adding Shards¶

Since configurable entities in the router can also be accessed via the management protocol, we can remotely add a shard to the above example using qdmanage:

qdmanage create --type org.apache.qpid.dispatch.connector host=<host> port=<port> name=broker3
qdmanage create --type org.apache.qpid.dispatch.router.config.autoLink addr=queue.first dir=in connection=broker3
qdmanage create --type org.apache.qpid.dispatch.router.config.autoLink addr=queue.first dir=out connection=broker3

Navigation

  • index
  • next |
  • previous |
  • Qpid Dispatch 0.6.1 documentation »
  • Qpid Dispatch Router Book »
  • 2. Using Qpid Dispatch »
© Copyright 2013, Apache Qpid <http://qpid.apache.org/>. Created using Sphinx 1.4.5.