JSON-RPC

We could keep POST /runs and invent our own body for each procedure. A standard format gives every call the same shape: the procedure’s name, its arguments, and then either a result or an error. Then one function in the application and one handler on the runner work for every procedure. Libraries for the format exist in most languages.

JSON-RPC is a standard protocol for remote procedure calls. Its current version, 2.0, was published in 2010. A call is a JSON object, the request object. What the procedure returns comes back in another JSON object, the response object.

JSON-RPC defines the two objects, and we choose how to send them. We send each request object in the body of an HTTP POST, to one endpoint on the runner, POST /rpc. Every call goes to that endpoint. GraphQL works the same way: every GraphQL request goes to POST /graphql.

Here is the application’s call to run Nadia’s code:

POST /rpc HTTP/1.1
Host: runner.codedrill.internal
Content-Type: application/json

{
  "jsonrpc": "2.0",
  "method": "runSubmission",
  "params": {
    "language": "java",
    "code": "public static boolean isBalanced(String s) { ... }",
    "tests": [
      { "name": "empty string", "args": [""], "expected": true },
      { "name": "nested", "args": ["([]{})"], "expected": true },
      { "name": "closed in the wrong order", "args": ["([)]"], "expected": false },
      ...
    ]
  },
  "id": 1
}

• jsonrpc is always "2.0". It says which version of JSON-RPC the object follows.
• method names the procedure, runSubmission. It is a member of the request object, not the HTTP method. The HTTP method is POST for every call.
• params holds the arguments. Here it is an object, so each argument has a name.
• id identifies the call. The application chooses it, and the response object carries the same id.

The host name runner.codedrill.internal can be reached only from inside the department’s network. Only the application calls the runner.

The runner runs the code and responds:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "jsonrpc": "2.0",
  "result": {
    "compiled": true,
    "tests": [
      { "name": "empty string", "passed": true },
      { "name": "nested", "passed": true },
      { "name": "closed in the wrong order", "passed": false },
      ...
    ]
  },
  "id": 1
}

result holds what the procedure returned. The application saves it with submission 8812 and puts it in its response to Nadia.

Why every call has an id

Over HTTP, the id looks unnecessary. Each HTTP response belongs to one request, so the application already knows which call a response is for. But JSON-RPC can also run over one connection that stays open and carries many calls, such as a WebSocket.

Recall when we considered posting Priya’s messages over the WebSocket in Courselore. A WebSocket message has no response, so we would have had to rebuild request and response ourselves. Each message would carry a number that the response repeats, and we would define our own error codes in place of status codes. JSON-RPC is that design, written down as a standard. The id is the number, and the error object, below, carries the codes. The responses can come back in any order, and the caller matches each one to its call by the id.

Results and errors

Suppose Nadia’s code does not compile. The runner still carries out the call: it compiles the code, and the compiler reports a mistake. Nadia needs to see the compiler’s message. So the runner returns it as a result, not as an error:

{
  "jsonrpc": "2.0",
  "result": {
    "compiled": false,
    "compilerOutput": "Line 4: missing return statement"
  },
  "id": 2
}

The same goes for a test that fails and for a run that is stopped at the time limit.

An error is for a call that the runner could not carry out. Then the response object has error in place of result:

{
  "jsonrpc": "2.0",
  "error": {
    "code": -32601,
    "message": "Method not found"
  },
  "id": 3
}

An error has a code, a number that a program acts on, and a message for a person. A GraphQL error has the same split. JSON-RPC defines a few error codes of its own. They are all large negative numbers, such as -32601 for a procedure that does not exist and -32602 for wrong arguments. A call to runSubmission without tests has wrong arguments. For our own errors, we can use integers outside JSON-RPC’s reserved range of -32768 through -32000. For example, we define the code 1001 for a call whose language the runner does not support, such as "language": "rust". The runner has a list of the languages it accepts, so any other value is a mistake in the call. An unknown term code was the same kind of mistake in the Public Courses API.

Status codes

JSON-RPC defines only the request and response objects, so the HTTP status is our choice. The runner could map each error to a status, such as 404 for a procedure that does not exist. But the application reads the body anyway, and the code in the body says more. So I send 200 OK with every response object. The outcome is in the body. GraphQL does the same for errors found while a query runs: the response is 200, and the client checks the body for errors.

The application can still receive a status other than 200. For example, the load balancer in front of the runner sends 502 Bad Gateway or 503 Service Unavailable when it cannot pass the call to a runner instance.