More Than One Server

One server is enough for Courselore. But if Courselore ever grew past what one server can hold, or if you design a system at that scale, WebSockets can run on several servers. It takes more work to manage.

Suppose Courselore ran on two instances, Instance 1 and Instance 2, behind a load balancer. Recall horizontal scaling for the Public Courses API: identical instances behind a load balancer, which can send any request to any instance. That works because the API is stateless.

An instance that holds WebSockets keeps state the other instances do not have: its own connections and the sets that list them. Suppose Marcus’s WebSocket connection is open on Instance 2. Priya posts a message, and the load balancer sends the POST to Instance 1. Instance 1 saves the message in the shared database and sends it on every connection it has for that room. Marcus’s connection is on Instance 2, so Marcus never gets the message.

The instances need a channel between them. Each instance sends every new message to the channel, and each instance delivers what it receives from the channel to its own connections in the room’s set. Redis, which we met as a cache server, is a common choice for the channel.

The load balancer needs work too. It must pass the opening handshake, with its Upgrade header, through to an instance, and it must keep connections open for a long time instead of closing them when they are quiet.

Courselore runs on one server, so it needs none of this for now.