On June 13 the video sphere stood on a road case at Machine Yearning, the 16th NYC Resistor Interactive Show: screens live, arms out, twelve viewpoints of one object. At its base, a phone played a Knicks game under a hand-taped sign begging to keep the game on.
This is the summary post. The introduction covers what the video sphere is: a discrete light-field display, twelve inward-facing screens on icosahedron vertices, each showing what a camera at that spot would see. This post is what got built, by the numbers, and what held it together.
The numbers
| Thing | Value |
|---|---|
| Screens | 12 × LCDWIKI ES3C28P (ESP32-S3, 2.8″ ILI9341, 240×320) |
| Per-device rate | ~16 fps sustained (30 ms decode + 30 ms DMA, serialized) |
| Host send rate | 20 fps from TouchDesigner |
| Frame payload | JPEG q55, forced 4:4:4, typically 4-10 KB with a 32 KB cap |
| Wire protocol | 3 packed structs, 3 UDP ports (8888 / 8889 / 8890) |
| Aggregate bandwidth | ~1.9 MB/s ≈ 15 Mbps on one 2.4 GHz HT20 channel |
| Printed parts | 63: hub, 13 stems, pedestal, 4 case parts × 12 modules |
| Firmware | Pure ESP-IDF 5.4, ~280 KB, one identical binary per board |
| Per-device config | Two bytes in NVS: device_index and screen_rotation |
Three contracts
A one-person build of a twelve-node distributed display stayed tractable because the system has exactly three boundaries, and each one is a contract small enough to hold in your head.
- TouchDesigner → wire. Twelve renders become flipped, quantized, 4:4:4-encoded JPEGs, fragmented into ≤32 chunks of 1024 bytes. Everything spherical (camera placement, up vectors, perspective) is resolved on this side; the wire never knows the screens form a sphere.
- The wire. Three little-endian packed structs, 8 to 16 bytes of header each. An 8-byte data header, an 8-byte broadcast sync, a 16-byte telemetry heartbeat. The whole protocol fits on an index card.
- Wire → display. Reassemble in a four-slot PSRAM ring, decode to an internal-SRAM back buffer, present when the sync says go.
Each contract is independently testable: tools/send_jpeg.py proves the entire firmware side with no TouchDesigner in the room, and TD’s sender was debugged against nothing but heartbeats. Swap any side (another engine for TD, Ethernet for WiFi, a different panel library) and the other two don’t move.
How it unfolded
The git history tells a compressed story. The first commit, April 20, is the spec, written as a build brief for an AI coding assistant, with packet layouts, memory budgets, and a bring-up ladder (one screen → two → four → six → twelve). Code lands four days later, and the first day of code ends with a five-device swarm running the full pipeline (receive, reassemble, decode, sync, heartbeat) on DHCP with auto-discovered IPs.
Then the character of the work changes. April 28 to May 3 is the hardware sprint: fourteen commits of enclosure corrections as printed reality contradicted modeled assumptions. May is power: deep sleep and the battery badge. Late May into June is connectivity hardening in the field, a BSSID lock because two APs shared one name, a retry policy of never giving up. June 8, a three.js visualizer for planning the physical assembly. June 13, the road case at Machine Yearning.
What I’d keep from the spec-first approach: the spec was wrong in places, and that was fine, because every deviation got recorded against it. Static IPs became heartbeat-discovered DHCP, the atlas TOP became twelve direct reads, and triple-sync became a single send with permissive receivers. The spec didn’t have to be right to be useful; it had to hold still, so the deviations stayed legible against it.
The series
- The Video Sphere: the introduction
- The Icosahedron Hub: modeled in OpenSCAD
- The Screen Enclosure: the per-screen cases
- The Printed Parts: printing everything
- The Hardware Revisions: the iteration log
- The Camera Rig: the TouchDesigner side
- The Screen Firmware: the ESP32-S3 firmware
Not finished, exactly
The status file keeps a list of what remains, so here it is. The full calibration session (walking the assembled sphere, verifying every screen’s ID, hue, and up-arrow against its mounted position) hasn’t happened. The RSSI survey tool is scaffolded but waits on the firmware to report signal strength in its heartbeat. The dedicated, isolated access point (fixed channel, no internet, static IPs) is still a plan; the sphere currently rides a shared network with a BSSID lock. And there’s a known ~33 fps sitting behind a second framebuffer nobody’s needed yet.
None of that blocked the thing that mattered: an object floating inside a ball of screens, holding its shape as you walk around it, while ten inches below a basketball game played on a phone.
Part 7 of the video-sphere series.