Lab notebook

Writing for an AI draftsperson

· Oct 22 – Nov 3, 2026

In February 2010 I took Casey Reas's workshop at Medialab-Prado in Madrid. His work starts from Sol LeWitt's wall drawings, where LeWitt wrote a plain-language instruction and other people drew it on the wall. Reas writes his text first and then makes software that performs it, so each run of the software is one reading of the same text.

This year I started working that way with AI models. I write the text and the models draw it. Early on I wrote this to Claude:

"ai fills ambiguity. so either i have to write eveyrthing down, shich is not feasable and also i doesnt work well with that. or divide it all smaller pieces and make meaning full pieces one at a time."

I wanted to find out which words in a text change what comes back. The first test was about giving the models real material, and the second was about the wording of my own text. I ran them in October and November, mostly with models from labs other than Anthropic, and I wrote down what I expected before each run.

Real material in the brief

On 22 October I wrote "i dont know it feels like if we know red is red why do we paint a pointing only in red - thats the feeling i have with ai". So I tested whether material from the world moves a model off its red.

I gave four models from four labs the kind of brief I'd type myself: "Make a quick sketch of this: an app that collects data from multiple resources and creates a data visualization." Some runs got only that. Others also got real material: 72 hours of tide readings from The Battery in Manhattan, or Barış Manço's discography with sources that don't agree. Four models, five versions of the brief and eight pages per model per version made 160 pages. Here is one page from each model for three of the versions.

One page per model for three versions of the brief
One page per model for three of the five versions of the brief. The words and names change by row; the layout stays with the column.

With only the brief, 18 of the 32 pages had the same generic page, a list of data sources with a "connect a source" button, usually named something like Gather or DataWeave. With the tide readings or the discography, that page almost went away; it showed up on 8 of the 128. The names changed to Tidegraph and Manço Data Lab, and the words on the page came from the material.

If you look down a column, the layout hardly changes. Each model kept its own layout whatever I gave it, and which model made the page mattered about three times more than the material did, so the material moved the words and the names and left the models' red where it was. The tide pages also came out dark navy, and I think the models just associate tide data with that colour.

Ten builds of the same text

The second test was on the text I'm writing for a tool I'm building. It describes a one-to-one working session between a person and an AI partner. It marks some things as fixed, for example that the AI partner never changes a part the person has locked, and leaves a few open on purpose. Five models each built it twice as a clickable prototype, choosing their own example project, and the second time they had to pick a different one. These are the first screens of all ten.

First screens of ten builds of the same text
Ten builds of the same text. Columns are models, rows are the first and second example project.

Three models from other labs read every build and recorded how it handled each of the text's 39 parts. For each part I counted how many different answers the ten builds gave. 0.1 means all ten handled the part the same way, and 1 means all ten handled it differently.

Average disagreement between the ten builds, by kind of part

Data
Value
Marked fixed (the text decided it)0.2
Marked open (left to the builder on purpose)0.275
Never marked (the text said nothing)0.325

The parts I left open on purpose varied more than the fixed ones, which is what I'd hoped, and the parts I never mentioned varied the most. Of the nine places where the builds disagreed most, six were parts I had written as fixed. The example project in the text also steered the builds. For three of the five models, building around a different example project changed how they handled the text almost as much as switching to a different model did.

Then I added a sentence to the text about when a part is locked. I changed two sentences in that run, so I can't fully separate their effects. Before the change 4 of the 10 builds locked parts the way I meant, and after it all 10 did. Three other decisions were in a separate decisions file. The builders followed them only if I gave them the file; for one of them it was 1 of 10 builds without the file and 9 of 10 with it.

How I write now

After these tests I put the actual readings, records and dates into the text and leave out the adjective that describes them. I don't expect the material to change the look, so I choose the layout some other way. I read the places I didn't mark before the ones I did, because that's where the builds went their own way. I write each decision as one plain sentence in the text, since the decisions file only reached the builders when I handed it over. I keep examples out unless I want every build to look like them.

The tests were small: one brief for the material test, measured by counting names, the generic page and colours, with no human ratings, and ten builds for the text. Claude ran the tests, and models from two other labs checked the write-up against the numbers before I read it.

Material test, 22 Oct 2026: Claude Haiku 5.5, GPT-6 Luna, DeepSeek V4.1 Flash and Qwen 3.8 Flash, 8 pages per model per version, $2. Text tests, 3 Nov 2026: builders GPT, Gemini, Grok, Kimi and DeepSeek, judged by the other labs' models. Tide readings from NOAA station 8518750. Reas's essay: {Software} Structures, Whitney Artport.

Next entryChanging the model or changing the brief