360 Replay at Super Bowl LI

FOX Sports joined with Intel to bring a new Player’s-Eye View to Super Bowl LI. This is not captured by Go Pros worn by players like Fox referee GoPro cams. This new system uses Intel’s 360 Replay technology. It includes 38 separate 5K cameras circling the playing field, each with its own dedicated server. Five miles of fiber optic cables connect it all to one control room, where six producers from Intel and one producer from Fox Sports determine and produce which 360 degree replays will be used in the broadcast.

This is an enormous amount of data – about 1 terabyte per each 15-30 second clip. This much data allows any point in time to be re-created in volumetric or 3D space. Any cubic inch on the field is captured in what they call a “voxel,” a contraction of “vox” for volume and “el” for element. Just like a virtual camera system that is created in 3D video games, a virtual camera can be created and positioned at the real football player’s eye line. At one point the producers even experimented with superimposing a face guard over the view, but that was rejected.

Fox Sports uses this capability to create what they call “Be the Player” which can generate a point-of-view anywhere on the field including a player’s POV on the field. TV viewers can then see what the player sees as well as the choices he had. The director chooses which player to cover. Depending on the play it might be a receiver, quarterback or other position. This is not instant replay because it takes a couple minutes to render. That’s why you see a 360 replay during time outs or other non-action periods.

In this year’s Super Bowl, there were three examples of 360 replay – two were “Be The Player” and one was a 360 view of a key play. They will probably be available on YouTube soon.

This system is not likely to be used in high school football. It took a month to retrofit the Houston stadium with all of the equipment needed for the data capture. This kind of data capture is called “freeD.” There is so much data that, like any 3D environment, it can be used to spin around and zoom. While this 360 replay system has been used in baseball, the Olympics, and the NBA, the Super Bowl was the first time it enabled TV audiences to see the game from the players’ POV what Fox calls “Be the Player.”

To promote its 360 replay technology, Intel uses it in an ad featuring Tom Brady in mundane everyday life. It demonstrates how this technology can make anything look epic.