On June 13 the video sphere stood on a road case at Machine Yearning, the 16th NYC Resistor Interactive Show: screens live, arms out, twelve viewpoints of one object. At its base, a phone played a Knicks game under a hand-taped sign begging to keep the game on.
This is the summary post. The introduction covers what the video sphere is: a discrete light-field display, twelve inward-facing screens on icosahedron vertices, each showing what a camera at that spot would see. This post is what got built, by the numbers, and what held it together.
The numbers
| Thing | Value |
|---|---|
| Screens | 12 × LCDWIKI ES3C28P (ESP32-S3, 2.8″ ILI9341, 240×320) |
| Per-device rate | ~16 fps sustained (30 ms decode + 30 ms DMA, serialized) |
| Host send rate | 20 fps from TouchDesigner |
| Frame payload | JPEG q55, forced 4:4:4, typically 4-10 KB with a 32 KB cap |
| Wire protocol | 3 packed structs, 3 UDP ports (8888 / 8889 / 8890) |
| Aggregate bandwidth | ~1.9 MB/s ≈ 15 Mbps on one 2.4 GHz HT20 channel |
| Printed parts | 63: hub, 13 stems, pedestal, 4 case parts × 12 modules |
| Firmware | Pure ESP-IDF 5.4, ~280 KB, one identical binary per board |
| Per-device config | Two bytes in NVS: device_index and screen_rotation |
Three contracts
A one-person build of a twelve-node distributed display stayed tractable because the system has exactly three boundaries, and each one is a contract small enough to hold in your head.
- TouchDesigner → wire. Twelve renders become flipped, quantized, 4:4:4-encoded JPEGs, fragmented into ≤32 chunks of 1024 bytes. Everything spherical (camera placement, up vectors, perspective) is resolved on this side; the wire never knows the screens form a sphere.
- The wire. Three little-endian packed structs, 8 to 16 bytes of header each. An 8-byte data header, an 8-byte broadcast sync, a 16-byte telemetry heartbeat. The whole protocol fits on an index card.
- Wire → display. Reassemble in a four-slot PSRAM ring, decode to an internal-SRAM back buffer, present when the sync says go.
Each contract is independently testable: tools/send_jpeg.py proves the entire firmware side with no TouchDesigner in the room, and TD’s sender was debugged against nothing but heartbeats. Swap any side (another engine for TD, Ethernet for WiFi, a different panel library) and the other two don’t move.
How it unfolded
The project starts April 20; code lands four days later, and the first day of code ends with a five-device swarm running the full pipeline (receive, reassemble, decode, sync, heartbeat) on DHCP with auto-discovered IPs. Bring-up climbs a ladder: one screen → two → four → six → twelve.
Parts of the original design got replaced along the way: static IPs became heartbeat-discovered DHCP, and triple-sync became a single send with permissive receivers.
The series
- The Video Sphere: the introduction
- The Icosahedron Hub: modeled in OpenSCAD
- The Screen Enclosure: the per-screen cases
- The Printed Parts: printing everything
- The TouchDesigner Camera Rig: the TouchDesigner side
- The Screen Firmware: the ESP32-S3 firmware
Future work
The status file keeps a list of what remains, so here it is. The full calibration session (walking the assembled sphere, verifying every screen’s ID, hue, and up-arrow against its mounted position) hasn’t happened. The RSSI survey tool is scaffolded but waits on the firmware to report signal strength in its heartbeat. The dedicated, isolated access point (fixed channel, no internet, static IPs) is still a plan; the sphere currently rides a shared network with a BSSID lock. And there’s a known ~33 fps sitting behind a second framebuffer nobody’s needed yet.
None of that blocked the thing that mattered: an object floating inside a ball of screens, holding its shape as you walk around it.