How to tell if your agency’s code is any good, without reading it

You hired an agency because you can’t build it yourself. Which creates the awkward loop every non-technical founder knows: the people telling you the work is good are the people you are paying to do the work.

You cannot escape that loop by learning to read code — by the time you can review a pull request, you didn’t need the agency. But code quality, like kitchen hygiene, leaks into things you can observe from the dining room. Here are five tests that require no technical knowledge, only attention. None of them is conclusive alone. Two or three failing together is a pattern, and patterns arrive earlier than disasters.

1. Working software, or artefacts about software?

Healthy teams show you the thing. Every week or two, you should be clicking through the actual product — rough, part-finished, occasionally broken, but real and cumulative: this week contains last week.

The warning sign is a diet of artefacts instead: decks, mockups, “sprint summaries”, timelines, a demo that is always one week away or always driven by the same person on the same laptop over a screen-share you can’t touch. Teams demo working software when they have it. When you are consistently shown descriptions of the thing rather than the thing, it is usually because the thing does not currently work — and the distance between you and the product is being managed.

Ask for access, not a walkthrough: a link you can open on your own phone on a Tuesday night. The reaction to that request is itself the test result.

2. The small-change test

Ask for something trivial: a copy change, a moved button, a new line in a form. Genuinely trivial — ask them to confirm it’s trivial, so nobody can later claim it was secretly hard.

Then watch the clock. In a healthy codebase with a healthy pipeline, a trivial change reaches you in hours to a couple of days, because the path from “edit” to “live” is paved: tests run themselves, deploys are routine, nobody is scared of the code. If a confirmed-trivial change takes weeks, you have learned something important: either the codebase is tangled enough that nothing is trivial, the deployment process is manual and frightening, or your work is queued behind other clients. All three are your problem now.

This is the single highest-signal test on the list, and it costs you one sentence.

3. Listen to the questions they ask you

Good engineers interrogate the problem. Early on, you should be fielding questions that feel almost annoyingly specific: what happens when two people edit the same record, what should a half-completed signup see when they come back, which of these fields is legally required, what happens at zero results, who else needs access to this data and why?

Those questions are what edge-case thinking sounds like from outside — the same instinct that, inside the code, becomes error handling and data integrity. If the only questions you ever hear are about screens and colours — or worse, if you hear no questions at all, just cheerful agreement with everything you say — then the difficult thinking is either not happening or being done by assumption. Assumptions ship. They surface later as “ah, we assumed X”, at which point X is load-bearing.

A useful prompt: ask them what the hardest part of your project is. A team that names something specific and slightly boring (“the data migration”, “handling payments failing mid-flow”) is thinking. A team that says “nothing too tricky!” about a system that moves money is not.

4. Ask to see it fail

Ask, in a demo: “Show me what a user sees when something goes wrong — bad input, no connection, a payment that doesn’t go through.”

Mature work fails on purpose: clear messages, a way back, nothing lost, somewhere a log that lets an engineer see what happened without asking the user to describe it. Immature work fails by surprise — blank screens, raw error text, a spinner that never resolves — because every hour went to the happy path that demos well.

You are not really testing error screens. You are testing whether the team builds for the product’s real life, where users mistype and networks drop and third parties time out, or only for its stage life. The happy path is perhaps a fifth of professional software; you are checking the other four-fifths got built.

5. The walking-away test

Ask this early, in writing: “If we parted ways tomorrow, what would I have, and could another team pick it up inside a week?”

The good answer is specific and slightly dull: the code lives in a repository you own or can access, there are instructions a competent developer could follow to run it, the accounts — hosting, domains, app stores, third-party services — are in your name, and the answer to “what would another team need?” is a short list, not a shrug. The bad answers are recognisable at any distance: “it’s all handled”, hosting you’ve never seen a login for, anything resembling you wouldn’t need to leave.

This is also the cleanest proxy for internal quality you have. Code that a team is happy for strangers to inherit next week tends to be code they are not ashamed of. Lock-in and mess travel together, because both are what happens when nobody expects to be checked.

Before you’ve signed: the same tests, earlier

Four of the five tests have pre-contract versions, and they are cheaper to run before the deposit clears.

Ask every agency you’re considering for a link to something live they built — not a case study page, the actual product, which you then open on your own phone. Ask their references the small-change question directly: “when you asked for something trivial, how long did it take to go live?” Reference calls are usually a parade of pleasantries; that one question produces numbers. Ask to see a handover pack from a finished project — redacted is fine; what you’re checking is that one exists and that “what the next team needs” is a real document rather than a promise. And ask, in the pitch, “what’s the hardest part of our project?” — the team that names something specific and slightly boring is the team already doing the thinking you’re paying for.

You will notice the pattern: at every stage, the test is never the answer itself but how the question lands. Teams with nothing to hide treat these as normal professional questions, because they are. Discomfort with scrutiny, this early, at this stakes level, is the free sample of the relationship.

If two or more of these fail

Don’t start with accusations — start with a second opinion. A few days of an independent senior engineer reading the codebase and the setup will tell you whether you’re looking at normal mid-project scruffiness or a structural problem, and crucially it converts the conversation from feelings (“we’re worried”) to findings (“deploys are manual, there are no tests around payments, the hosting is in the agency’s name”). Sitting on the client’s side of that table — for a few days, or a day or two a week — is a large part of what fractional CTO work actually is. Every competent agency will tolerate this; the reaction of an incompetent one is, once again, the test result.

What these tests can’t tell you

Being honest about the limits: these are tests of process health, and process health is a proxy. It is a good proxy — rot on the inside almost always leaks out through cadence, latency, and handover — but a proxy can miss things. A team can demo weekly, ship small changes in hours, and still be quietly accumulating an architecture problem that only a technical reader would spot; that is exactly the case the second opinion exists for, and why it’s worth a few days of an independent engineer’s time at least once in a long engagement, even when everything looks green.

The tests can also false-positive. A two-person studio mid-crunch might take a week over your trivial change and still be doing excellent work; a big agency’s polished handover pack can dress up mediocre code. That is why the standard is two or three failing together, and why the pattern matters more than any single result. What the tests reliably catch is the thing you can least afford: a team whose relationship to scrutiny is defensive. Everything else about an engagement can be fixed cheaper than that.

And if all five pass — say so, pay promptly, and hold on to them. From the client side of the table, teams like that are rarer than the day rate suggests.

Drizzlelabs is an independent software studio in Bristol — the tools in the footer are what its own work looks like shipped, and the about page has the longer story.