What Native AOT actually costs

Start with the good half, because it is real

Nona is a configuration service. It should be the least interesting thing in your infrastructure - small, quiet, always up, never the reason you are awake at 3am. Native AOT is enormously well suited to that. Our AOT image idles at 61MiB of process RSS and serves a hundred-key config to fifty concurrent clients at 1,617 requests a second while peaking at 123MiB. It is a single self-contained binary in a chiseled container with no runtime to install.

For self-hosted software that somebody else has to operate, that profile is worth a great deal. We are not writing this to talk anyone out of it. We are writing it because the arrangement has another half, we did not read the terms closely enough before agreeing to them, and all three parts came due at the worst possible moment.

Cost one: you cannot profile what you ship

We went looking for a memory problem. The first thing you reach for in .NET is a heap dump, so we reached for dotnet-gcdump, pointed it at the Native AOT build, and it declined. There is no diagnostics server in there to talk to. The thing we ship had gone somewhere we could not follow it.

Sit with that for a second, because it is worse than an inconvenience. The artifact we ship to production is the one artifact we cannot introspect. So we did the only thing left: built the same source framework-dependent and brought that up into the daylight to be examined instead. Different GC configuration surface, different startup path, an entire runtime present that is absent in production. A stand-in wearing the same face.

Every conclusion we drew about memory came with an asterisk: we measured the understudy. Most of it transferred - allocations live in application code, and application code does not care how it was compiled - but we never once closed the loop on the thing our users actually run. And this is not a toll you pay at the gate and forget. It comes due again every single time you need to look inside. If your service is ever going to have a memory mystery, price that in now.

Cost two: AOT put HTTP on our read path

This is the expensive one, and it took us a while to see the chain.

Nona can run primary/replica for multi-region deployments. A replica keeps its own copy of the data and serves reads locally, so a client in Tokyo talks to the Tokyo node instead of crossing an ocean. libSQL supports exactly this via embedded replicas - your process holds a local database file that syncs from a primary, and reads are plain local reads.

Except we do not do that, and here is why:

if (options.EnableLocalReplica)
{
    throw new NotSupportedException(
        "Storage:Libsql:EnableLocalReplica is not supported by the AOT libSQL client. " +
        "Use a managed sqld replica with --primary-grpc-url instead.");
}

The native libSQL client was a companion we could not bring down with us, so we hand-wrote an HTTP client that speaks the wire protocol instead. Our project file does not reference a libSQL package at all. Which means every read on a replica leaves our process, goes over HTTP to a sqld sitting in the same container, and comes back as JSON:

Roughly four milliseconds of flat overhead per read, on localhost, for data sitting on the same disk. And be careful how you read that table, because we got it wrong ourselves at first: four milliseconds to avoid a hundred and fifty millisecond ocean crossing is a trade you take every single time. The replication architecture is right. It is the last inch that annoys us - the replica already has the file locally, and we ask for it over HTTP anyway.

That inch is AOT's bill, not replication's.

Cost three: you cannot build it where you write it

We develop on Windows and ship Linux containers. With a normal .NET build that is a non-event. With AOT:

error : Cross-OS native compilation is not supported.

Native compilation needs the target platform's toolchain, so a Linux binary has to be produced on Linux. You cannot make the thing from up here. You have to go down and build it where it is going to live - in practice a Docker build with clang and zlib installed, and the native compile step alone takes about a minute before anything else happens.

It is entirely workable - our Dockerfile has done it from day one. But notice that this is the same shape as the profiler problem, for the third time: the thing you iterate on and the thing you ship are two separate objects, and they never meet. Half the year up here, half the year down there.

The smaller change you make without noticing

AOT also quietly rewrites how you code. No runtime reflection means source-generated JSON everywhere, and a serializer context you have to remember to feed every time you add a type. Plenty of otherwise sensible libraries are simply off the menu.

We happen to like this. It pushes you towards explicit, boring, inspectable code, which is the kind we want in a service that is supposed to be uninteresting. But it is a constraint on every future decision, not a one-time setup cost, and it is the sort of thing that is invisible when you are choosing AOT and obvious eighteen months later.

Would we do it again

Yes. For a self-hosted configuration service, 61MiB idle and a single binary with no runtime to install is worth more than any of the three costs above. Somebody else operates this software, on their metal, under their memory limit. Small and quiet wins.

What we would do differently is read the terms before eating. The pattern in all three is one pattern: you get size and speed, and you give up the ability to reach in after it. No profiler on the production binary. No native client that would have kept reads local. No build on your own machine. None of it appears in the benchmark that sold you on the idea, and all of it appears on the day something goes wrong.

So take the twenty-millisecond startup. Ours is a settled arrangement now and we plan around it the way you plan around a season. Just budget a day for the afternoon you need a heap dump and find that what you shipped is somewhere you cannot go.