info.bogdan.ws

It’s a pretty familiar loop. You pick a problem, write some C++ in a Monaco editor, then hit Run to try it against your own input or Submit to face the hidden tests. A few seconds later you get a verdict for every test, one of AC, WA, TLE, MLE, RE or CE, plus how long each one took and how much memory it used.

That is the core loop. Around it there is an admin area for writing problems and uploading test data, plus a dashboard that shows how you are improving over time. My favourite part is a debugger that lets you step forward and backward through a run while it shows what each variable held on each line.

I started out thinking sandboxing mostly meant setting a timeout and a memory cap. It didn’t take long to realize that wasn’t enough.

Running strangers’ code on your own server is an adversarial problem. A submission can go read files, open sockets, fork until the machine crashes, or just sleep forever without burning any CPU. Each of those needs its own defense, and if you miss even one, it’s only a matter of time before the judge has a really bad day.

What I actually wanted was simple to say and hard to build, which meant verdicts I could trust and timing numbers I could believe, no matter what someone submitted.

The stack is simple on purpose. It’s Go with Fiber on the backend, PostgreSQL and Redis behind it, and a React 19 SPA built around Monaco on the front. Everything runs in Docker except the one service that actually executes user code. That’s where all the interesting work lives.

The executor is a small standalone Go service running natively under systemd. It has to live outside Docker because isolate, the sandbox used at the International Olympiad in Informatics, wants full cgroup v2 delegation, and Docker’s cgroup namespace won’t give it up. So the backend reaches it over host.docker.internal.

isolate does most of the heavy lifting: a fresh PID, network, and mount namespace for every run, cgroup v2 caps on CPU time, memory, and process count, and a read-only root filesystem.

I pass every limit as an explicit flag, and the sandbox refuses to start at all if cgroup support is missing. A judge that quietly stops enforcing memory limits is worse than one that’s just down.

args := []string{
	"--cg",                // apply limits to the whole box
	"--processes=20",      // allow compiler child processes
	"--quota=102400,1000", // 100 MB and 1000 inodes
	"--fsize=104857600",   // 100 MB file limit

	fmt.Sprintf("--mem=%d", cfg.MemoryKB),
	fmt.Sprintf("--time=%.2f", cfg.CPUSeconds),
	fmt.Sprintf("--wall-time=%.2f", cfg.CPUSeconds*3),
	fmt.Sprintf("--meta=%s", metaPath), // return time and memory
}

On top of that there’s a seccomp-BPF filter. Go assembles the allowlist and compiles it to BPF, then a tiny C launcher installs it at the last possible moment. The ordering is everything. The filter has to be in place before the untrusted binary runs a single instruction, and the process has to give up the right to gain privileges first, or a setuid binary could shed the filter on its way in.

if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0)
	die("no_new_privs");

struct sock_fprog prog = { .len = filter_len, .filter = filter };
if (prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog) != 0)
	die("seccomp");

execvp(argv[i], &argv[i]);
die("execvp");

I whitelist compiler flags, and every test case gets a brand-new box, so nothing one test leaves behind can leak into the next.

One small detail I still like is that clang’s ANSI colors survive the whole trip back to the browser, so a compile error looks just like it would in your own terminal.

I spent longer than I’d like to admit trying to make isolate happy inside Docker. Every workaround fixed one thing and broke another. Limits would stop being enforced, timings would come back wrong, or the container needed so many privileges that the sandbox stopped meaning anything.

In the end I stopped fighting it and put the executor on bare metal. It’s a slightly uglier deploy, with two process managers and two log streams, but the isolation actually holds, which is kind of the whole point.

Right now the executor judges submissions one at a time on a single machine. For the traffic it gets, that’s completely fine. If I ever ran a proper contest with a few dozen people all hitting submit at once, I’d turn the executor into a pool of workers instead of a single process. The design already leans that way, but I just haven’t needed it yet.