Making Animal Vision, a camera that sees like other animals

When I later wrote the idea down for a rebuild, it read like this:

"I want to see how animals see the world. For example a fly, then I want to change that to a dog, etc. I want to use Google Cardboard as my VR tool, so the view needs to be in VR. It needs to be scientific data-based, not cartoonish."

The first version was made on March 27, 2026, in one Replit Agent session. The fly came first: the live camera passed through a shader that breaks the image into compound-eye facets, pushes the colour toward blue and green and removes red. About half an hour later the agent had added a mantis shrimp, a honeybee and a dog. Seventeen minutes after that, eight more animals from Wild Kratts. In New York time the fly was in at 10:03 in the morning and the app was published before 1 pm.

"Not cartoonish" stayed the rule. Two of the agent's tasks that first day were only about scientific accuracy. One fixed the dog, where the agent had called the green-sensitive cones "red-sensitive". The other was a review of all 14 animals, which the agent reports as checked against published research.

Then the Wild Kratts idea grew into the app. As I wrote it, the user "can say "Creature Power" to activate camera scanning of creature disks (Wild Kratts term)." The agent connected Gemini 2.5 Flash to the camera, so you can hold up a disc or a picture and the app switches to that animal. If Gemini names an animal the app doesn't have, it researches that animal's eyes, picks the closest shader, and saves the new profile. Because the phone sits inside the headset, voice became the controls: "Creature Power!" scans, and "What If" records 30 seconds of the plain camera and the animal view side by side.

That evening I added animal hearing filters. The microphone runs through a filter chain built from each animal's audiogram, the curve of what it can hear. The agent added a warning to wear headphones, because the speaker feeds back into the microphone.

I also tried other versions in Replit: two fly-only Cardboard pieces that put you in a 3D garden at fly height instead of using the camera, and a start on a native phone app with Expo.

Rebuilding it from a book

In April I tried something different. On April 11 the rules behind the app were written out as ten "riffs" in a book in Lades, a tool I was building then, plus a nine-move "tune" for the whole app. On April 12 I gave three new Replit agents the same file, which says "The book is the spec. This file is not the spec," and told them to search the book before every decision.

All three built the easy moves, like the dark interface and stat chips. None built the move the book called the most important: never dead-end on an animal the app doesn't know. Where the book had no numbers, each agent wrote its own, and they disagreed: one build gives the honeybee and the dragonfly opposite flicker rates from the other two. In this test the rules were not enough; the book also needed the facts and numbers.

Turning it into a game

In September I started turning it into FieldVision, built with Replit Agent and Claude Code: a creature-collecting game for kids 8 to 12. The brief I pasted into Replit starts: "A real-world creature collecting game for kids who live in Roblox and Minecraft. Go outside, find a real animal or plant, and unlock its power: see the world through its eyes." One of the game rules is that after about 60 seconds of creature vision the game says "Now look with your own eyes."

I directed the look from screenshots. On one round I wrote "i saw the C screenshots. it is better," and then "C, but still I think there are too many lines and corners there." The calmer version became the design rules.

What changed most between March and September was how the app treats generated animals. In March they appeared in the grid looking exactly like the curated ones, on purpose. In FieldVision, scanning is a secondary, experimental path, generated animals are labelled experimental, and the app says on screen that a phone camera can't capture ultraviolet, polarised or thermal light. More about the finished app is on the project page.