The Broker Makes the Copies

The broker already holds the email queue and the shipping queue. So instead of the order worker sending two messages, the broker can make the copies:

• The order worker sends one message for order 93 to the broker.
• The broker puts a copy on the email queue and a copy on the shipping queue.

Then the order worker does not need to know the services or their queues.

Which queues get a copy

The broker has to know which queues get a copy. It keeps a map from a name to a list of queues:

order-placed → email queue, shipping queue

The order worker sends the message for order 93 with the name order-placed. The broker looks up order-placed in the map and puts a copy of the message on each queue in its list.

Topic

Each key in the map is set up in the broker ahead of time. The order worker cannot send a message with a name the broker does not have. A key set up this way is called a topic. Here, the topic is order-placed, and its list holds the queues that get a copy of each message sent to it:

graph TD
    W[Commerce service: order worker] -->|order 93| T
    subgraph B[Broker]
        T[order-placed topic] -->|copy| EQ[Email queue]
        T -->|copy| SQ[Shipping queue]
    end
    EQ --> N[Notification service]
    SQ --> Sh[Shipping service]

The order worker names only the topic. It no longer names the email queue or the shipping queue.

Subscribing

Each service still has its own queue in the broker. A service tells the broker which topics it is interested in, and the broker adds the service’s queue to the list of each of those topics. The notification service is interested in order-placed, so the email queue is in the topic’s list. The shipping service is interested in it too, so the shipping queue is in the list as well.

Telling the broker that a service is interested in a topic is called subscribing to the topic. Here, the notification service and the shipping service subscribe to order-placed.

In code

Here is the same design with RabbitMQ, from Node.js with the amqplib library. RabbitMQ calls a topic an exchange. Each service opens a connection to the broker, and in the code below, channel sends commands to the broker over that connection. The code leaves out opening the connection and handling errors.

The MQ in RabbitMQ stands for message queuing. RabbitMQ uses a protocol called AMQP, the Advanced Message Queuing Protocol. The name amqplib comes from AMQP.

Setting up the topic

The topic is set up in the broker once, ahead of time:

await channel.assertExchange("order-placed", "fanout");

The second argument, fanout, tells RabbitMQ to put a copy of every message on every queue in the topic’s list.

Subscribing to the topic

The notification service creates its queue and adds it to the topic’s list:

await channel.assertQueue("email");
await channel.bindQueue("email", "order-placed", "");

assertQueue creates the queue if it does not exist yet. bindQueue adds the queue to the list of order-placed. RabbitMQ calls an entry in a topic’s list a binding. Other kinds of RabbitMQ exchange use the third argument to choose which messages reach the queue. A fanout exchange copies every message to every queue in its list, so it ignores the third argument. The empty string is a placeholder.

The shipping service does the same with its own queue, shipping. Each service does this once, not once for each message.

Sending a message

The order worker’s message for order 93 carries the shop and the order:

const message = { shopId: "mugshop", orderId: "93" };

To put a message on a queue, as the commerce service does with the order queue, a program names the queue:

channel.sendToQueue("orders", Buffer.from(JSON.stringify(message)));

To send the message to a topic, the order worker names the topic instead:

channel.publish("order-placed", "", Buffer.from(JSON.stringify(message)));

The topic is an argument of the call, not part of the message. The broker reads the topic, looks up its list, and puts a copy of the message on each queue in it. A fanout exchange ignores the second argument, the empty string. Buffer.from turns the JSON text into bytes, because RabbitMQ sends bytes.

As with the order queue, the broker confirms the message only after the copies are saved to disk and copied to a second server.

Receiving a message

The notification service takes messages from its own queue. It does this the same way a consumer takes messages from any queue:

channel.consume("email", (msg) => {
  const topic = msg.fields.exchange;  // "order-placed"
  const order = JSON.parse(msg.content);  // { shopId: "mugshop", orderId: "93" }
  // send the email for this topic and order
});

The broker delivers each copy together with the name of the topic it came from. The message itself does not contain the topic’s name, so the notification service reads order-placed from msg.fields.exchange. Real code also tells the broker when it has finished with each message. We leave that out until a later chapter.

The two services take their copies independently, and neither one waits for the other. So the notification service may send the email before or after the shipping service starts the shipment.