When a Call Fails

The client stub makes a call to the runner look like a local function call, but the call still goes over the network. That gives a submission a new way to fail: the application can send the code to the runner and receive no response. The runner may be down, still working, or finished with a response that never reached the application. We have to decide how long Nadia waits and what happens to the submission if no results arrive.

How long to wait

CodeDrill’s submission response time requirement is 10 seconds. The application cannot spend all of that time waiting for the runner: it also needs time to read the problem, save the results, and respond to Nadia.

I budget at most 8 seconds for the runner and leave 2 seconds for the application’s other work. Reading the problem and saving the results take a few milliseconds when the database is not busy, so 2 seconds leaves room for a slow moment. The application gives the call a deadline: the time at which it stops waiting. If no response arrives by then, the call has timed out. Stopping the wait does not by itself stop the work on the runner.

The runner stops a run at 5 seconds. Starting and removing the container add about 1 second, compiling adds about 1 more, and the network round trip takes about a millisecond, because both services are in the same network. So a run that reaches its time limit returns its results within about 7 seconds, and the 8-second budget allows it.

Sending runSubmission again

Recall the client on a phone that sent POST /notes and timed out. Sending the request again could create a second note, so we needed an idempotency key. A missing response from the runner leaves the same uncertainty: the first call may have completed before we send another.

We design runSubmission so that repeating it creates no duplicate records or other lasting effects. The application saves the submission and results; the runner only computes them. So repeating the call is safe even if the first call completed.

If the call fails quickly, for example because the runner instance it reached had stopped, the application tries once more within the original 8-second budget. It does not restart the clock. If the deadline has passed, or the runner reports a mistake in the call’s arguments, the application does not retry. Repeated attempts consume more runner capacity, so retries must be limited even when they are safe.

What happens to Nadia’s submission

If no results arrive, we have two choices. We can report the failure and ask Nadia to submit again. Or we can save the submission as pending and arrange to run it later. The second choice preserves Nadia’s submission through a runner outage, but the application must track unfinished work and let Nadia retrieve the results later.

For CodeDrill, I keep the first choice. The application reports that results are unavailable, asks Nadia to submit again, and does not save this attempt. This keeps the design simple, at the cost of making Nadia repeat the submission after a failure. Nadia gets an answer within 10 seconds, but the answer is a failure, not the results. So this choice does not fully meet the submission response time requirement. If CodeDrill must accept submissions during runner outages, we would need to revisit this choice and the requirement that results arrive in the same response.