The 75 Years of Innovation series highlights the groundbreaking innovations spanning from SRI’s founding in 1946 to today. Each week, SRI will release an innovation, leading up to its 75th anniversary in November 2021.
Setting the standards for augmented reality for broadcasting.
The ‘First-Down’ Magic Marker: How Augmented Reality Changed the Game
“I can’t think of another digital innovation that has had as much impact on the fan viewing experience as that yellow line.” — Jim Delaney (an NFL fan for more than 35 years)*
When you watch sports on the TV screen, all may not be as it seems. Take the first down marker, the yellow line that seems to move with the game like magic. What’s that all about?
The yellow marker line, which gives so much information about the game, looks to all intents and purposes to be there, in the game, as it plays out. But it is placed there using Augmented Reality, added in during play using clever technology.
Sports fans do not like impediments when watching sport, but they do like information. If you interfere with the match too much, push back from the fans ensues. In 1996, according to the book “The Billion Dollar Game: Behind the Scenes of the Greatest Day in American Sport” this is exactly what happened with a new technology known as the “FoxTrack Puck” or ‘technopuck’. The technology was used in Ice Hockey to place a glowing blue line around the puck that allowed viewers to track it as it moved over the ice. Fans did not like the technopuck and it was dropped after the first game.
This rejection of the technopuck was duly noted by the tech and broadcasting industry. If you want to augment a game you need to do it really well; so well, that it is noticeable, but feels integral. The result was the unobtrusive, Augmented Reality of the “first-down yellow line” which has become a feature of the game and which spawned the technology behind the virtual ads we see during games.
But what is L-VIS (Live Video Insertion System) and how does it work so well?
The Technology Behind Live Video Insertion System (L-VIS)
Like the game itself, the technology behind the first-down yellow line (L-VIS) is complex but beautiful; the whole being greater than the sum of the parts.
The first preparation step carried out before the game is the creation of a 3D mathematical model of the playing field. Part of this process is to generate a map of the colors of the field at the time of play — taking light conditions into account. The colors of the player uniform are also mapped. As we will see, color mapping is a very important part of the L-VIS that adds ‘magic’ to the yellow line.
During play, three cameras are used to pan across the field at all angles, generating data in real time. Computer software and specialized algorithms then combine this data alongside the 3D model, analyzing the mapped coordinates to draw the line in the exact position for the game. This is redrawn showing the correct perspective for any given frame as the game proceeds.
The augmented reality of L-VIS gets really neat when an object, such as a player, gets in the way of the line. The computer must remove the line overlaid on the object; otherwise, it would look like the yellow line was painted across the object. The color mapping comes into play at this point. The mapped colors are coordinated to the yellow line which is removed by the software and replaced with the right colors. This gives the impression that the line is underneath any object in its way.
The result is a magic yellow line that moves as the game plays on.
And, more magic, the game is only delayed by one second when all of this is happening.
The Place of VAI and L-VIS in the History of Technology
On September 27, 1998, ESPN’s Sunday Night Football broadcast carried the first-down yellow line.
Back in the early 1990s, Brown Williams was a senior manager at David Sarnoff Research Center (now part of SRI International). Williams had been part of a team working on pattern recognition and tracking technology for various U.S. defense agencies. Commercializing this methodology was floated as an idea and a contract was placed with Sarnoff to develop a prototype. This was delivered in early 1994. The innovation developed through this prototype was in the creation of pattern recognition algorithms that fitted to the exacting needs of live-action TV and the expectations of consumers.
The subtle use of Augmented Reality can add massive benefits to marketers both on and off the playing field. The L-VIS technology is used widely and can also be seen during play in the form of ads. An example being the virtual TV screen shown during the FOX broadcast of Super Bowl XXXIII; people watching the game from a TV seeing an image that was not visible to those in the stadium.
Today, Augmented Reality (AR) is part of many everyday marketing experiences. International companies such as IKEA, use it to allow customers to place products directly in a room to check the fit before purchase. Digi-Capital predicts that AR will have an install base of around 3.5 billion generating revenue of up to $90 billion by 2022.
The world according to Augmented Reality is here and it is real.
Resources
*Mashable, The Yellow First-Down Line: An Oral History of a Game Changer: https://mashable.com/2013/09/25/yellow-first-down-line/
Press release 1996 “ABC/ESPN TO UTILIZE VIDEO INSERTION TECHNOLOGY THIS SPRING”: https://www.sportsbusinessdaily.com/Daily/Issues/1996/02/23/Sports-Media/ABCESPN-TO-UTILIZE-VIDEO-INSERTION-TECHNOLOGY-THIS-SPRING.aspx
Vox video on how the first-down yellow line works: https://www.youtube.com/watch?v=1Oqm6eO6deU
IKEA Augmented Reality app: http://www.ikea.com/gb/en/customer-service/ikea-apps/
Patent, Rosser, R.J., Williams, B.F., System and method for downstream application and control electronic billboard system: https://patents.google.com/patent/US5543856A/en?q=H04H60%2f20
