How AI agents built Veduta in 82 reviewed steps, and what on-device AI taught me

A team of AI agents built my app in 82 steps, each one reviewed before the next began. Then I put it on a phone and used it, which is where most of the learning happened. This post covers how the agents built it, what it cost, and what I learnt putting small AI models inside the app itself.

Veduta is a small idea. You pick a photograph, and a few moments later you get a row of designs: your words set into the picture like a magazine cover, clear of faces, in a colour you can actually read. You swipe between them, drag the text where you want it, and export a still or a short clip.

I did not type the app's code. I wrote down what it should do, read the plans and the reviews, and tested every build on a phone. A team of AI agents wrote the code. Those agents run in the cloud, on Claude. The app itself uses small AI models that run on the phone, so your photographs never leave it.

Below, "we" means me and the agents. The photos in the screenshots are AI-generated samples; no real people appear in them.

Three pieces: the app, its design engine, and the AI that built them

There are three pieces of software in this story.

• Veduta is the app, for Android and iOS, written once in Kotlin Multiplatform.
• Vinyas is the engine inside the app. It looks at the photograph, decides where the words can go, sets them in a typeface, checks they are readable, and draws the result.
• ASTRAMAN is the builder. It takes a written description of an app and turns it into a working app, one small step at a time, with a review after every step.

How ASTRAMAN turns a written description into an app

ASTRAMAN is a small team of AI agents. Each one is Claude with its own instructions and its own job.

Four decisions made the biggest difference.

Code checks the description, not an agent. The description is a set of plain files: numbered requirements, one file per screen with "when I do this, that happens" rules, colours and sizes. A script checks them before anything is built, and writes each step's own checklist of what "done" means. No agent ever has to guess.

Each skill is written down once. About forty short guides, called skills, cover things like navigation, the database, dark mode and accessibility. Library versions live only there. The plan never copies a version number, because a copied number goes stale the day the guide is updated.

Scripts do the repeatable work. Some steps produce the same files every time, like the empty project or the custom style checks. A script writes those in seconds, for free. The AI only steps in if the script fails.

The strongest model plans, a fast one writes code. Halfway through the Veduta build I moved the code writer to a newer, faster Sonnet. The code writer's typical (median) time per step halved, from about 13 minutes to about 6.

Building Veduta in 82 reviewed steps, then testing it on a real phone

Veduta's plan had 82 steps, from an empty project to ready for the store. (The cost log below has 81: one step left no entry in it.) It used the real engine from the first step, with no fake data behind the screens.

Every step passed review: it built, its tests passed, and it did what the description said. The next check was the one no review can do: installing the app on a phone and using it.

That found four things worth fixing in the first few minutes: a button that froze, settings from one photograph showing up on the next, a drag that lagged behind my finger, and a move that did not stick. Each was quick to fix once seen. The reviews had checked the code against the description; only using the app showed how it felt.

So a review and a phone are two different checks, and every build now gets both. Reviews keep the code honest; the phone keeps the app pleasant to use. It keeps paying off: this week a new tap-to-write feature passed every test, and on the phone the words came out near-black on a dark patch of jungle, hard to see. One look at the screen caught it, and the fix went in the same day.

What the build cost in time, tokens and money

ASTRAMAN records every agent run: how long it took, how long it waited for Gradle (the tool that compiles an Android app), how much text the model read and wrote, and what that would cost on the API. The model counts text in tokens, roughly three quarters of a word each. Here are the numbers for Veduta and for two earlier test builds of a Sudoku app.

A few things stand out.

• Checking costs almost as much as building. The reviewer cost US$240 and the code writer US$281. In return, it sent work back 27 times before a problem could carry into the next step.
• The time goes to the AI, not the compiler. Waiting for Gradle was only 16% of the time. Every speed-up that mattered came from the AI side: a faster model, less to read, scripts instead of prompts.
• The agents mostly re-read what they have seen before. For every new token, about ninety thousand came from the model's short-term memory (the cache): the skills, the description and the code from earlier steps. That is what keeps a long build affordable.
• A faster model was the simplest speed-up. Switching the code writer at step 30 halved its median time per step and cut its median cost from US$4.29 to US$1.81. The whole step, review included, went from US$7.25 to US$4.72.

The costs are what Claude Code reported for each run, at API list prices. I ran the builds on a Claude subscription, which is why there are pauses for usage limits. The token counts do not include text written into the cache, which the log does not record. Two things are not in these numbers. The design work, done with Claude Design over three rounds, used well over a million more tokens that I did not measure. And none of it counts my own time: writing the description, the design rounds, reading reviews and testing on a phone.

Why Veduta keeps every photo on the phone

Veduta's first rule: no photograph ever leaves the phone. There is no server, no account and no tracking. It works offline from the first launch.

So everything the app knows about a photograph comes from small AI models that run on the phone itself.

The Android models are open source and come to about 8 MB together. iOS uses what Apple already ships with the phone.

The models only answer "where are the faces, where is the subject, which areas are light or dark". Everything after that, where the words go, how big they are, which colour reads best, is ordinary code. It takes well under a quarter of a second per design.

Lesson 1: We removed the AI that wrote captions

The first version also had a language model on the phone: Apple's on-device model on iOS, with Google's Gemini Nano considered for Android. At first it wrote a caption for the photo. Then five captions to choose from. Then we asked it where the words should go and what colour they should be.

We tested that last idea on five photographs.

• Where to put the words: right once in five. It put them right where a person was holding a piece of coral, after being told never to cover the subject.
• What colour: it said "dark" for all five, including one that needed light words on dark leaves.

A language model cannot measure how readable a colour is on the real pixels. Ordinary code can. So the rule became: the AI says what it sees, the code decides.

Then the captions got slower and no more useful. On iOS at the time, looking at the photo took 0.2 to 1.4 seconds, but writing five captions could take another 18 seconds. Worse, Apple's built-in safety filter refused an ordinary photo of a man lifting a dumbbell in a gym ("May contain sensitive content"), so the app spun for most of a minute and showed nothing.

An early tester put it bluntly: the designs were great, the captions were the worst part. So we removed the language model. A full set of designs now appears in under two seconds, and people write their own words.

Use AI to describe what is in the picture. Use ordinary code for anything you can measure.

Lesson 2: Cutting the photo into squares helped the iPhone find small faces

Veduta has to know where the faces are, so it never puts words over them. On iOS it asks Apple's Vision framework.

When a face is small, under about 3% of the photo's width, Vision misses it if you hand it the whole photo at once. On a photo of a football game in a park, it found none of the seven players' faces:

Our tests had not noticed. They compared the engine's answer with a saved answer for each photo, and those saved answers had been recorded from the same detector. Both said "no faces", so the test passed.

So I marked every face in the 39 test photos by hand, 128 in all. Against that, whole-photo Vision found 38, and only 3 of the 75 small ones. The fix was to cut the photo into overlapping squares, look for faces in each, and merge the results. On the two group photos it went from 0 of 14 faces to 14 of 14.

Check a detector against answers a person has checked, not against what it said last time.

Lesson 3: We compared ML Kit with BlazeFace, and kept both

On Android there are two common ways to find faces: Google's ML Kit, and a small open model called BlazeFace. Here they are on a busy flower market, with the hand-marked faces in the bottom-right panel:

Across five crowded test photos, against the hand-marked faces (a face counts as found when a box covers at least half of it; run on an Android emulator):

It is not a clean sweep. On a pool party and a ferris wheel at dusk, ML Kit found more faces, though on the ferris wheel it drew 34 boxes for 11 faces. Stray boxes matter here, because every box is a place the words must avoid. BlazeFace with the smaller squares found the most faces with the fewest false alarms, so it is the default. The smaller squares take about 25% longer, which is worth it.

Finding the main subject was a harder call. ML Kit drew better outlines, but on a real Pixel it crashed the whole app with a native crash, which the app cannot catch and recover from. The engine's first rule is that it never crashes the app, so the open DeepLab model is the default and ML Kit is off unless you switch it on.

I kept both options in the code. A future update to either model could change the answer, and switching back is one line.

Lesson 4: The slow part was our own code, not the AI models

Reading a photo on Android took about 3 seconds. Now it takes about 1, and a large 12-megapixel photo went from 5.6 seconds to 1.4. The results are exactly the same as before.

None of the speed-up came from the AI models.

• The three models now look at the photo at the same time, instead of one after another.
• The squares for the face detector are cut before it asks for them.
• The code that shrinks the photo before the models see it, plain Kotlin, was more than half of the total time. Rewriting it was the biggest win.
• Each model starts warming up when the app opens. The first photo after launch used to be about half a second slower than the rest; warming up removes most of that.

ASTRAMAN taught the same thing. Before trying to speed up builds, I measured where the time went on an earlier Sudoku build: about 81% was the AI thinking and only 19% was compiling. Veduta's build looked the same, at 16% compiling. Speeding up the compiler could never have helped much.

One more rule from the app: never make the screen wait for a model. Veduta shows your photo straight away and the designs fade in underneath as they are ready.

What I tried that did not work

• Running several agents at once. I tried building several steps in parallel. It was no faster, and their changes clashed when merged. A faster model for each step did far more.
• Telling the agents in words. When an agent made the same mistake twice, I first added a sentence to its instructions. The mistake came back. What worked was a check in code that fails the build. When "keep each commit under 300 lines" became a check instead of a sentence, oversized commits across ASTRAMAN's builds went from about 30% to under 1%.
• Taking an agent's word for it. An agent once said a failing test was "already broken before my change". It was not. Now I check claims like that myself.

How we work now: build in small pieces, test everything once

• Agents work on small, separate pieces at the same time, and only run the quick checks: style checks, does it compile for both phones, and the tests for what they touched. A few minutes each.
• When the pieces are merged, one full run checks everything: every test, the iPhone simulator tests, the screenshot tests, and the full iOS build.
• Only when that passes does anything get published, all at once.

Since the 82 steps finished, this way of working has added layouts for foldables and iPads, in-app purchases, several texts on one design, and a row of text styles. Each style shows as a small "Aa" drawn by the engine itself, so what you tap is exactly what you get.

Most of these changes came from using the app, not from the description. The location used to sit inside the headline. Now it is its own text with a pin, which you can move or delete like any other. And the app no longer makes you write a line first. Every tool is there from the start, and you can tap anywhere on the photo to write.

What I still do myself

ASTRAMAN builds the app. It does not decide what the app should be. These stay with me:

• writing the description and working through the design rounds;
• product calls, like removing the captions or keeping phones in portrait;
• testing on a real phone, with sample photos only;
• everything in the app stores: products, prices, signing and legal text.

I keep a note of every time I had to step in during a build. It is one of the most useful files in the project. Each entry is either a call only a person should make, or a gap in ASTRAMAN that I should fix in code so I never have to step in for it again.

If you want to build apps this way

• Write down what the app should do as structured files, and check them with code before any agent reads them.
• Keep versions and how-tos in one place, never copied into plans.
• Let scripts do anything that comes out the same every time.
• Plan with the strongest model, write code with a fast one.
• When a mistake repeats, add a check, not a sentence.
• Review every step, and still use the app on a real phone before you trust the review.
• Inside the app, let AI describe the picture and let ordinary code make the decisions.

Veduta is not in the stores yet. What is left needs a person: store listings, signing and the legal pages. When it ships, I will link it here.

The common thread: real evidence beats polish. Every lesson above came from looking at the actual screen, the actual photo, or the actual numbers, not from a passing test or a confident summary.