Testing AI-Written Code Without Fooling Yourself

Previously, in Part 8, staff chat taught us the architectural lesson of the series — the circuit is a SignalR connection. ClinicLive now books, checks in, calls next and chats. Which brings us to the part most AI-assisted projects skip, right before the part where they regret it: tests.

Why AI code needs tests more, not less

There's a tempting syllogism floating around: the AI wrote it, we reviewed it, it works in the browser — so testing it is bureaucracy. We'd argue the opposite, for two reasons that are specific to how this app got built.

First: when you write code by hand, you accumulate scar tissue as you go — a private memory of every edge case you worried about, every almost-bug you caught mid-keystroke. That memory is a real, if unreliable, safety net. We reviewed every file the AI wrote (Part 5's rule), but reviewing gives you breadth, not scars. We didn't live through writing this code, so our intuition about where it's thin is thinner too. Tests are how we buy back the confidence we'd normally have earned the slow way.

Second, and more important: every prompt from here on is a change made by something that doesn't remember writing the code either. When we ask the AI to fix a bug in Part 10 or harden things in Part 11, nothing in its head is protecting the booking flow. A regression harness is that protection. Tests aren't a tax on AI development — they're the thing that makes the next ten prompts safe.

Here's the prompt, verbatim from the commit message in the companion repo (tag part-09):

"Add tests/ClinicLive.Tests. Unit tests for the confirmation-code
alphabet and the slot calendar. Integration tests against a REAL PostgreSQL
via Testcontainers (postgres:18, one container per collection): booking
creates patient+code, double-booking the same slot fails via the partial
unique index, cancelling frees the slot for rebooking, check-in creates the
queue entry, and a test pinning the waiting-list order. No mocking library —
QueueService only touches hub.Clients.Group().SendAsync(), so hand-write a
30-line no-op IHubContext."

Two decisions in that prompt do most of the work, so let's take them one at a time.

A real database, not a look-alike

Our most important business rule — two active appointments can never share a slot — isn't enforced in C#. It's a partial unique index with a PostgreSQL filter string, living in the database (Part 6's whole point: the database is the last line of defense). An in-memory provider or SQLite stand-in doesn't understand that filter, so a test suite built on one would happily pass while testing nothing we actually rely on. If the defense lives in Postgres, the tests run against Postgres.

Testcontainers makes that nearly free: the fixture starts a genuine postgres:18 container — the same image our docker-compose uses — runs the real migrations against it, and throws it away afterwards. One container serves the whole test collection:

public sealed class PostgresFixture : IAsyncLifetime
{
    private readonly PostgreSqlContainer _container = new PostgreSqlBuilder()
        .WithImage("postgres:18")
        .Build();
    // ...
}

[CollectionDefinition("postgres")]
public class PostgresCollection : ICollectionFixture<PostgresFixture>;

The tests are a few hundred milliseconds slower than fake-database tests. In exchange, a green run means the code works against the engine we ship. That trade is not close.

The 30-line fake that replaced a mocking library

QueueService takes an IHubContext<QueueHub> so it can broadcast QueueChanged after every change. Tests don't care about broadcasts — the real-time side got verified in the browser back in Part 7 — they care about database rules. The reflex here is to reach for a mocking framework. The prompt forbade it, and the result is one of our favorite files in the repo:

/// <summary>
/// QueueService only calls hub.Clients.Group(...).SendAsync(...) — a no-op stand-in
/// is 30 lines and needs no mocking library. The real-time side is exercised in the
/// browser; these tests care about the database rules.
/// </summary>
public sealed class FakeQueueHub : IHubContext<QueueHub>
{
    public IHubClients Clients { get; } = new NoClients();
    public IGroupManager Groups { get; } = new NoGroups();

    // ...NoClients and NoGroups return NoProxy, and NoProxy does this:
    private sealed class NoProxy : ISingleClientProxy
    {
        public Task SendCoreAsync(string method, object?[] args,
            CancellationToken cancellationToken = default)
            => Task.CompletedTask;
    }
}

Every method returns a completed task; nothing is recorded, nothing is verified. Sometimes the simplest fake is no library at all — thirty lines you can read in one breath, with no setup DSL and no versioning treadmill. Save the mocking framework for when you actually need to assert on interactions.

War story: the model lives in DI

First run of the suite, and EF Core refused to migrate: "The model for context 'ApplicationDbContext' has pending changes." Confusing, because the app itself migrated that same database happily. The diagnosis took a genuinely pleasant AI conversation, and it's worth retelling because the lesson generalizes.

In Program.cs, the app configures ASP.NET Identity's store schema version through dependency injection. That option doesn't just tweak runtime behavior — it shapes the EF model. Our tests built their DbContextOptions from scratch, without that Identity configuration, so the model EF constructed in the test process didn't match the migration snapshot — and EF, quite correctly, refused to proceed. The fix is in the fixture:

// The app sets Identity's store schema version through DI (Program.cs), and
// that setting shapes the MODEL. Without it, the test-built model wouldn't
// match the migration snapshot and EF refuses to migrate ("pending changes").
var identityServices = new ServiceCollection()
    .Configure<IdentityOptions>(o => o.Stores.SchemaVersion = IdentitySchemaVersions.Version3)
    .BuildServiceProvider();

var options = new DbContextOptionsBuilder<ApplicationDbContext>()
    .UseNpgsql(ConnectionString)
    .UseSnakeCaseNamingConvention()
    .UseApplicationServiceProvider(identityServices)
    .Options;

The lesson, AI or no AI: your EF model is a product of configuration, not just of your entity classes — and a test environment must mirror every piece of configuration that shapes it. If your app tunes options in DI, your fixtures need the same tuning, or you're testing a subtly different database than the one you run.

The tour: what eight green tests actually prove

The suite came out at eight tests. Here's what each one is really pinning down:

The double-booking test deserves a look, because it's the whole Part 6 design decision under glass — two bookings race for the same slot, and it's the database that says no:

var first = await service.BookAsync("Test Patient B", "+00-1111-0002", null, slot);
var second = await service.BookAsync("Test Patient C", "+00-1111-0003", null, slot);

Assert.True(first.Success);
Assert.False(second.Success);           // the partial unique index fired
Assert.Contains("just taken", second.Error);

And its mirror image — cancel the appointment, and the same slot books again, because cancelled rows leave the index's filter. Together they pin down the rule and its escape hatch. No fake database could have told us any of this.

The trap: a test that can only agree

Now the centerpiece. The prompt asked for "a test pinning the waiting-list order." The AI did what any diligent assistant does when a prompt underspecifies: it went and read the implementation. GetSnapshotAsync orders the waiting list by CheckedInAt — first come, first served. So the AI wrote a test that books a 15:45 and a 15:00 appointment, checks the 15:45 patient in first, and asserts that they're first in the queue:

[Fact]
public async Task Waiting_list_is_ordered_by_check_in_time()
{
    // NOTE (Part 10 will revisit this test): it asserts what the CODE does today.
    // ...books 15:45 for "Test Patient G", then 15:00 for "Test Patient H"...

    // The late-slot patient checks in FIRST.
    await queue.CheckInAsync(lateSlot.Appointment!.ConfirmationCode);
    await queue.CheckInAsync(earlySlot.Appointment!.ConfirmationCode);

    // ...

    Assert.True(lateIndex < earlyIndex, "first to check in is first in the queue");
}

It passes. Green tick, suite done, everybody's happy. And it is worthless — worse than worthless, because it now certifies the behavior it observed. A test derived from the implementation can only ever agree with the implementation. If the code has a bug, the test enshrines the bug; refactor the code correctly and the test will punish you for it. We call this the tautological-test trap, and AI assistants fall into it readily, because "read the code, write assertions that match" is exactly the kind of plausible-looking diligence they excel at.

The principle worth framing: tests come from the spec, not from the code. The code is the thing on trial; it doesn't get to write its own alibi. When you ask an AI for tests, point it at the spec, the requirements doc, the user story — anything but the implementation — or at minimum, review every assertion by asking "says who?"

Is first-come-first-served actually what ClinicLive's spec says the queue should do? We left the note in the test on purpose, and we're leaving the question open on purpose too: that test has a date with Part 10.

The harness is built, and it's already holding one piece of counterfeit confidence. Next time, reception files a complaint, a clinic east of Greenwich wakes up in the wrong day, and we debug both bugs with the AI from symptom to green test — Part 10: Debugging with AI — two real bugs, start to finish.