Reach
Workflow integration.
Real-time cost alerts in Slack and CLI. Anomalies surface where engineers already work.
4phases
LoRA training, image inference, datasets and captioning, video
01 · Visionary · Generative studio
One file is the whole studio.
Training, images, video and datasets in a single application that installs with one command and runs on your own GPUs. I designed it, engineered it and deployed it.
2model families
Wan and Krea, added without a second backend
1command
from clone to a running URL, on your own GPUs
Role
Design engineer — sole designer & builder
Scope
Product · UX · Python · written with an AI pair
Stack
Modal · ComfyUI · PyTorch · serverless GPU
Status
Deployed and running · open source
The source is on GitHub.
The switch lives inside the prompt field rather than in the chrome, because which one you get is a property of what you’re making, not an address you navigate to.
The problem
Three subscriptions and a terminal window to make one video.
Weights in one service, the interface in another, datasets in a third. Each one a login, a bill and a format. Making the thing is the smallest part of the day.
Install
Nothing to configure.
No secrets, no CLI setup, no account. The one token it needs is pasted into the UI and stored where the app can reach it.
Deploy
One command, one URL.
modal deploy app.py builds the images and returns an address that is the entire application — interface, API and GPU jobs.
Run
Your compute, your storage.
Weights land on your own volume, chosen explicitly. Nothing downloads on its own, and the folder layout is the contract — datasets are images with text beside them.
The console · try it
The picture holds the screen. The bar rebuilds under it.
Options live in a bar under the picture, never a rail beside it — a settings column costs the image 384 pixels of the one dimension it cannot get back, and vertical is the cheap axis. So the only question the strip is ever allowed to ask is how little of it can be true at once. Four models across two media genuinely differ; rather than one panel with half its controls quietly inert, the bar is assembled from what the chosen model says it reads. Change anything below and watch what does not move.
a red paper boat drifting across rain-flooded pavement, slow push in, overcast daylight
Wan 2.2 A14B
16:9
720p
5s
Seedrandom
H100
Two experts · silent
Generate
Negative prompt
Samplereuler
Schedulersimple
Steps20
CFG3.5
Shift8.0
Expert switchauto
A control that is present but ignored is worse than one that is absent — it is the interface making a promise the model will not keep.
The pass that bought the canvas back
380 pixels, returned to the picture.
Four LoRAs used to cost a row each — a select that wrapped, two number boxes apiece, 380 pixels of canvas for four filenames. They are written into the prompt now, <lora:name:0.8>, Automatic1111’s notation because it is what anyone who has trained these models already types. A fifth costs the picture nothing, and the sentence can finally say the thing a row never could: where in it the LoRA applies.
The same pass spent pixels in the other direction, and that is the part worth admitting. “Copy is a last resort” had been running the strip — design, then an icon, then words — and the hyperparameters are where I pushed it past what it can carry. 32 is a rank, an alpha, an epoch count or a seed with equal plausibility; an icon is a rebus for a word you already know, so it cannot tell you which number you are looking at. Someone who had trained these models for five years still had to hover every field. The rule was not dropped, it was scoped: a control that shows its own value gets no label, and a bare number is not a value.
Before the GPU hours
Most training runs fail in the dataset.
Not in the trainer. A caption set that repeats itself teaches the repetition, and you find that out eight hours and a GPU bill later. So the dataset screen counts what the trainer is actually going to read.
80 of 80 carry the trigger word, 80 of 80 are captioned, median seventy words. The list on the right is every clause appearing more than once — even lighting five times, with soft five times. That is the overfitting warning, delivered before the run instead of after it.
Captions are written by a vision model as prose, not tags, because the text encoders these models use parse grammar.
The decision nobody sees
Image and video were never two places.
The decisions that matter most leave no trace. There is no video mode to enter, no project type to choose at the start, and nothing announcing that two systems were joined — one canvas, one prompt, one gallery. Done properly it is unremarkable, which is why it has to be claimed here and nowhere in the product.
What differs between image and video is only the options. The canvas holds the screen either way — options sit in a bar under the picture, never a rail beside it, since a settings column costs the image 384 pixels of the one dimension it cannot get back.
01 · Nav rail — 232px
Tools
Models
Datasets
Train
Image
Video
Gallery
Six items: the backend’s own organisation wearing a UI.
02 · Settings rail — 384px
Generate
Wider, and permanent: options parked beside a picture you only sometimes change.
03 · Shipped — a bar underneath
Vertical is the cheap axis. The picture takes it all back.
The same photograph in all three, drawn to scale. A rail beside the canvas cannot be paid for out of anything but the picture, and the middle frame is the version that argued the options deserved it — it lasted until I killed it. The nav rail is in the repository’s history; the settings rail is reconstructed, because it never survived long enough to be committed. What replaced both: Generate is the page, with no nav item because it is not a place you go, and Train is one door on the right that doubles as the readout on the run, since a run lasts hours and you are meant to leave and keep working.
The same argument, run forward. Everything above takes chrome away where the model already allows it; the panel that remains is there because generation still takes long enough to need a staging area. When the latency goes, the panel should go with it — a gesture-driven canvas where nothing is labelled and everything is live, and you touch her face to change her. What the machine invented is marked differently from what you actually said, and rerolling it costs nothing.
Speculative, and deliberately labelled as such. What makes it a design rather than a mood board is the list underneath it.
How it stays empty
A veto list, not a manifesto.
Eleven sentences, each written to kill a specific thing when the cheap fix gets proposed. If a sentence has never vetoed anything, it isn't earning its place — cut it.
The failure to guard against isn't the model, or latency, or scope. It's month four, when something doesn't fit cleanly and the cheapest fix is a panel.
Where design engineering stops being a title
Two lines inside the model, so the canvas could take regions.
Regional prompting is live — different prompts over different parts of one canvas, as columns or rows you add and weight. It sits under Advanced, and the direction and the region rows stay hidden until the checkbox is on. The existing extension for it could not be made to work — not configured wrong, architecturally unreachable. Krea 2 is single-stream: text and image tokens run through one self-attention pass, so there is no cross-attention to patch, and switching the extension on was a silent no-op that concatenated the prompts and regionalised nothing.
The dead end
It assumes an attention that isn’t there.
Couple duplicates a text key/value bank per region and blends on the way out — a cross-attention move. Single-stream has no attn2 to reach. The feature would have to be rebuilt, not ported.
The cheaper answer
Mask the attention instead.
Concatenate every region into the text span, then bias the scores so an image token only sees the text of the regions covering it. Same result as Couple, one forward pass instead of N, no duplicated batch. The blocks already threaded a mask down to the kernel; upstream just passed None.
What it costs
Priced, and only when used.
An explicit mask drops off the flash kernel and the bias is quadratic in sequence — about 70 MB at 1024px. Both are paid only when regions are actually in play, so a plain prompt costs nothing.
Where it lives follows what it costs. Under Advanced, beside the negative prompt and the sampler — and nested once more inside that, since a direction and an empty region row are meaningless until you have said you want regions at all. Almost every generation is one prompt over the whole frame; a control that applies to the rare case should be present for the rare case and absent the rest of the time. The patch itself is two lines in one vendored file, and with no region builder present the behaviour is byte-identical to upstream. That is the whole bet: the smallest possible change to somebody else’s model, at the one place it was already prepared to accept it.
It also decides what the images may install. The fast attention kernels assert the mask is empty, so the inference image deliberately ships without them — while the video image, whose models never pass a mask, ships with them and gets the speed. And because an upstream sync would overwrite the patch and let regional prompting fail quietly, the smoke test asserts the patch is still there. A capability this thin has to be guarded by something that shouts.
Under the surface
Never rent a GPU to do CPU work.
Scale here isn't requests per second. It's dataset size, model size and cost per job — so the decisions that matter are about what runs where.
1file
interface, API and GPU jobs
50%
less weight memory, quantized to FP8
0
accounts, secrets or CLI setup
Working with an AI pair
The specification is the deliverable.
A pair that writes faster than you can read moves the whole job upstream: the constraint is no longer typing, it is deciding. So the artifact I maintain is not the code — it is the 192-line document at the root of the repo that the pair reads first. Three principles in priority order for when they conflict, then the conventions that settle the arguments they don’t.
CLAUDE.md · conventions
“Comments explain why, not what. Every non-obvious line in this codebase earns its comment by naming the failure that produced it. If a comment could be deleted without losing a fact, delete it.”
What it produced — 36 comments in the stylesheet alone
app.py · the header
“Generate and Train are not peers: one is where the machine is used, the other is where it is changed, and a 50/50 toggle asserts a balance that does not exist.”
app.py · an empty-state rule
“.blank, not .empty — the placeholder is .blank, so this rule matched nothing and the global svg{width:100%} inflated the glyph to the width of the canvas. A selector that misses is invisible in the CSS and enormous on the page.”
A rule the pair can apply without me is worth more than a review I have to run. This one turns every fix into a written post-mortem, so the reason survives in the place it applies rather than in a thread I would have to find again.
Thirty commits, seven days — the subjects are the decisions
- e92c324Rebuild the page around the canvas, and give video a home in it
- 9100fe5Send people to controls that exist
- fa5f0e1Measure the canvas fit instead of guessing at it
- 9f199a9Point the caption panel at human mistakes, not model phrasing
- e697738Make missing-model errors self-diagnosing
- 17a79b6Drop the migration script, which can only fail for anyone else
The other half of the method is the loop. The real UI is a Modal ASGI app, so seeing a CSS change meant an image build, a volume mount and a cold start — minutes, per edit. tools/preview_ui.py serves the same markup locally against stubbed JSON, and the stubs are shaped to hold the awkward states: a missing model, an uncaptioned dataset, a prompt too long to belong in a gallery card. The pair is only as fast as the feedback you give it, and the states worth designing for are the ugly ones.
Design philosophy
The best design leaves no trace.
The work I’m proudest of here is the work nobody will notice — two systems joined so completely that no one thinks to ask when they were separate. Good design is unobtrusive — Dieter Rams. Underneath sit three words that settle the arguments, in priority order when they conflict.
Antifragile
Errors diagnose themselves.
A missing model prints the volume, the exact path it wanted and what is actually there — the three facts that separate a wrong profile from a typo. Any error you can hit twice should have explained itself the first time.
Scalable
Keep the polled thing small.
Job records carry filenames; bytes are served off the volume by their own route. A dictionary polled every two seconds must never grow with the size of the result.
Future-proof
Don't build a second way to do the first thing.
Adding a second family of video model added no backend — it reuses the container, the warm process and the same job contract. What is per-family is a graph builder and a row in a table.
Why it exists Old photographs in motion
Two years of diffusion research. The first thing it made was a gift for my father.
Take 01 / 05
Play
Direction