It's two computers now

I owe you a correction before I owe you anything else.

In the July post I wrote that the whole pipeline runs on one machine sitting on my desk, and that everything has to fit on a single consumer graphics card. Both sentences were true when I wrote them. They aren’t now. It runs on two computers, and I should have said so sooner. This is the last of four notes from the stretch of work after that post, and it exists because I don’t want the old sentence sitting there unanswered while I talk about what I built.

The first machine is the one I meant: a Windows box with an RTX 5080, 16 gigabytes of video memory. That card is still where pictures get drawn and voices get synthesised. Sixteen gigabytes isn’t a round number I picked for the prose. It’s the ceiling every image decision in this project bangs into. Models get quantized down to 4-bit so they fit. (Quantizing means storing the model’s numbers at lower precision, trading a little quality for a lot of memory.) Two heavy programs can’t share the card without spilling into system memory and crawling. If you’ve wondered why a local pipeline is this fussy about which agent runs when, that’s why. There’s one fast card, and it’s small.

The second machine is a MacBook Pro with an M2 Max and 32 gigabytes of unified memory. I brought it in because I wanted the language models to be larger, and because 16 gigabytes is a cramped place to host both a painter and a reviewer.

What I hoped it would be, and what it actually is.

I’ll be honest about the hope, because otherwise this reads like I always knew. I hoped more memory would mean better pictures. A bigger model, higher precision, fewer of the identity collapses I wrote about in the first note. Buy RAM, buy quality. That’s a very engineer instinct, and here it’s wrong.

On paper, the Mac can run the same image models. In practice it would run them at 4-bit too, same as the 5080. Of those 32 gigabytes, the GPU only gets to use roughly two thirds, and this generation of Apple chips has no hardware for the 16-bit number format a lot of image code assumes. So there’s no quality upgrade on offer. You’d get a slower second painter, at the same precision. And the one thing that might actually fix faces, training a small adapter on one character’s face, still needs an NVIDIA card, which means the 5080.

I’m glad I looked this up before I moved the Painter. I’d have spent a week porting a workflow to buy myself the same pictures, only slower.

What the Mac is good for is language.

The Reader and the frame reviewer are both “read this, make a structured decision” jobs. They want a bigger brain more than a faster brush. I put a 35-billion-parameter model on the Mac and compared it with the 14-billion one I’d been running on the 5080. On a 5-chapter slice, the bigger model had 0 mislabelled lines against 3, 0 stretches of dialogue dumped into narration against 1, and a longest segment of 214 characters against 343, for 12% more wall-clock time: 34.8 minutes instead of 31. Better structure for a slightly longer wait. That part I’ll take any day of the week.

But this is the same model I mentioned in the note about the book, the one that quietly copy-edited the novel while it segmented it. Better structure, and a text that wasn’t quite the book anymore. I only get to keep the upgrade because the pass that snaps every segment back to the exact source runs after it. A bigger brain is fine. A bigger brain without a guard would have been a step backwards dressed up as progress.

The reviewer that looks at finished frames lives on the Mac too, so it no longer has to take turns with the Painter on the 16-gigabyte card. That sounds small. It isn’t, once you’re generating a chapter of pictures and then asking another model to look at all of them. Two jobs that used to be a queue now run on two boxes.

So the Mac is a second card. It isn’t a better card for the thing I was stuck on.

“It all has to fit on one consumer GPU” is still the interesting constraint. It’s just no longer the only computer in the room.

I want to keep the spirit of the July sentence, because the spirit is still the project. No cloud. No per-frame API bill. No model I can’t run in this house. The license post I promised in July is still coming, and it’s still the reason I won’t reach for a pretty weights file with a nasty license attached. None of that changed when I opened a laptop.

What changed is that “local” doesn’t have to mean “one process on one card.” It can mean two machines on a desk, each doing the job the other is bad at. The GPU box paints and speaks. The Mac reads and judges. Now when I catch myself saying a job is blocked by the hardware, I have to ask which of the two boxes I mean.

It didn’t touch the dead end I actually care about. Three brothers in one frame still render as fewer than three brothers. That isn’t a memory problem. It’s the three-reference-slot problem from the first note, and more RAM on either machine doesn’t add a slot. Bringing in the laptop didn’t shorten the demo. It shortened the wait on the Reader, and it let me review frames without kicking the Painter off the card. Those are real. They aren’t a clip.

Why bother writing this, if the demo still isn’t up?

Because I already published the old sentence, and because “we added a machine” is the kind of update that’s easy to dress up as progress when it’s mostly logistics. I’d rather say the unflattering version: I added a second computer, it made the language side better, and it didn’t get me the video I went quiet to make.

That’s all four notes in one line. I vanished to make a demo. I spent two months directing agents at a book, a voice, a pile of pictures, and a second machine. The book is now actually the book. The voice is already mine, with a ceiling I’m not going to pretend I trained through. Chapter one plays with pictures, and I won’t sign them yet. The hardware is two boxes, and only one of them draws.

I still owe you why every model has to be Apache 2.0 or MIT. I planted that flag in July and I haven’t forgotten it. It’s next, once this set is out, because coming back from silence by skipping the thing I promised is how you end up going silent a second time.

Until then: no clip. I’d rather you hear that from me than from a gray rectangle I never replaced, or from a beautiful frame I wouldn’t put my name under.