Pac-12 Networks’ New Studio by Advanced Systems Group is the Second Largest SMPTE ST 2110 Facility on the West Coast
Advanced Systems Group (ASG) Designs and Builds New Broadcast and Production Studios Ready for Over 850 Live Events in 12 Month Timeframe
The Client
The Pac-12 Networks is an American sports-oriented digital and satellite television network. The network’s studio and production facilities are headquartered in San Ramon, California. Pac-12 Networks is the first and only sports media company owned by its 12 Universities.
The Pac-12 Network produces telecasts of roughly 850 collegiate events each year – 350 events on Pac-12’s national network, and 500 events carried on Pac-12 regional networks.
The Challenge
Some projects have deadlines, and then there’s the new Pac-12 Networks production studio in San Ramon, California. With its long-term downtown San Francisco lease ending in May of 2023, the overarching need to reduce operating costs, while refreshing the technical infrastructure, and the first football game of the 23-24 season starting in August 2023, Pac-12 Networks needed a design and integration partner. The partner had one year to design, build, and get on-air a SMPTE ST-2110 compliant, 42,000 square-foot broadcast and production facility fit for the production of over 850 live events each year.
After a lengthy search, they selected Advanced Systems Group (ASG) for the end-to-end design and build including five production control rooms, five audio control rooms, one studio, four software-defined production control rooms, a large graphics bullpen, a replay room, and a large central equipment room that would be the facility hub. ASG also supplied the acoustic design of the control rooms and studios, working in concert with Bay Area architectural firm, HKS San Francisco and RLS Acoustics.
After ten years at its previous location, the network’s technical infrastructure was aging. Using experience gained over years of operation starting from the network’s studio launch in San Francisco in 2012, Pac-12 Networks’ leadership was clear on what worked and what didn’t.
Lessons Learned
One of the things Pac-12 Networks had learned was that due to the hub-and-spoke model the network is based on, with Pac-12 as the hub and the member schools as the spokes, they needed more control rooms and fewer studios in the new facility.
Pac-12 Networks pioneered the hub-and-spoke model of broadcasting where audio and video are acquired at the sports venues and then backhauled to control rooms in San Francisco for broadcast. This model reduces onsite staff at the schools and minimizes setup times as well as reliance on costly mobile trucks. Pac-12 Networks utilized the universities’ existing Internet2 IP backbone for A/V transport from the school venues to the Pac-12 centralized production hub, creating the largest private IP network in the United States dedicated to sports.
“Efficiency and an IP infrastructure were key must-haves. They knew that over the next 10 years having an IP-based plant would become very important to their hub-and-spoke method of production.”
Dave Van Hoy, President of ASG
“They had a very specific design vision in mind for what they wanted the new facility to be,” said President of Advanced Systems Group LLC Dave Van Hoy. “They knew exactly the number of control rooms and operators, and what they needed in terms of replay, graphics, and editorial. Efficiency and an IP infrastructure were key must-haves. They knew that over the next 10 years having an IP-based plant would become very important to their hub-and-spoke method of production. It’s an incredibly cost-effective way to operate, especially with the range of sports programming schools now must support.”
The Solution
SMPTE ST 2110
What was created in an ambitious one-year timeframe is the second-largest ST 2110 facility on the West Coast.
SMPTE ST 2110, a suite of standards for video, audio, control, and metadata transport over IP networks, was released in 2017. “However, it’s still young in terms of functional deployment, and interoperability is still an issue,” said Van Hoy. “When you consider the number of vendors brought in and the engineering work that had to be done to ensure products could talk to each other, you start to understand how fast this all came together – especially with the lingering issues of slowed delivery dates left over from the pandemic.”
A critical part of an innovative design like this for a systems integrator is heading to the lab to ensure the hardware and software function properly in a real-world environment. While ASG did conduct lab tests, many of the products were not available when testing had to start, adding another wrinkle to a tight deadline.
Networking equipment has been particularly hard hit with delays. Vendors Arista Networks and Imagine Communications accelerated delivery dates to accommodate the rushed timeline. Pac-12 Networks selected the Imagine Magellan™ Control System for IP routing, control, and monitoring and Arista high-speed Ethernet switches. Imagine’s Magellan forms the 2110 orchestration backbone of the facility, while Arista’s switching and routing are the core transport components. There are many key vendors involved in the Pac-12 Networks studio build. Included among them are Calrec Artemis audio consoles, Grass Valley K-Frame video switchers, Ross XPression graphics, Evertz Dreamcatcher replay systems, just to name a few.
Meeting New Challenges
The multi-discipline design and coordination effort required by ASG was one of the more challenging aspects of the project, according to Van Hoy. There were a variety of entities to collaborate with, including architectural and acoustics firms, the general building contractors, and various trades. The ASG teams were split in a similar fashion: one engineering group focused on the core 2110 layout from a systemic point of view, another group dedicated to all the subsystems, as well as a team concentrated on the physical layout.
Michele Ferreira, ASG Vice President, Systems Integration, was the overall lead on the entire project, working in concert with ASG Regional Sales Leader Garrick Huey, who acted as account manager leading most of the business discussions with Pac-12 Networks. Garrick also managed the vendor relationships during the entire project. Steve Young, Managing Director, Technology Consulting at ASG, served as Director of Engineering throughout the project.
ASG’s Engineering Director, Systems Integration, East, Nik Kumar, led the core IP transport 2110 system layout. The ASG acoustics team became the primary coordinator of the physical layout and furniture design, with ASG Principal Acoustical Architect Erika Lisman’s team running point with Randy Sparks, Principal of RLS. ASG Senior Project Manager Harrison Gardanier was the overall project manager, responsible for delivering all aspects of the project to Pac-12 Networks.
The ST 2110 protocol has many layers, including basic signal transport, protocol, and control pieces. Each component – audio, video, control, and metadata – is synchronized to each other while remaining an independent data stream. The signal transport layer is the most mature, while the coordination of control protocols is less so. For the ASG team, this required a very thorough commissioning process.
Van Hoy reported that the coordination of the audio portion of the 2110 infrastructure (AES-67) was an unexpected challenge in the Pac-12 Networks buildup when combined with Calrec’s sophisticated DSP and redundancy structure. It required some reconfiguration of the IP network. “In the baseband world, redundancy involves more wires, more equipment, and a switch to cut between things,” he said. “In the IP world, it means multiple engines running simultaneously within a networked digital environment, and how the software is going to cut between them. Implementing audio redundancy within an IP infrastructure is not a trivial undertaking. We got caught in a couple of places where those became very interesting challenges. Everything worked out, with many thanks to the impressive cooperation of the Calrec, ASG, and Imagine teams. It was definitely a learning lesson.”
“Working in baseband would have been much safer, especially given the timeline,” said Van Hoy. “We’ve been doing it for decades. There’s little risk to it. But, they were betting on a better future. This is transformative technology.”
A Facility for the Future
The decision to go with an ST 2110 compliant facility versus a baseband one was based on Pac-12 Networks’ forward-thinking approach from the start. “Working in baseband would have been much safer, especially given the timeline,” said Van Hoy. “We’ve been doing it for decades. There’s little risk to it. But with ST 2110, they were betting on a better future. From their centralized production point of view, the idea of capitalizing on IP-based transport all the way through, including the production endpoints, is transformative technology.
“With a control room on premise and the natural extension of 2110 infrastructure, it can become an automatic part of a university’s infrastructure. Because the Pac-12 Network only services large universities, it can take advantage of the remarkable level of connectivity on and off their campuses for high-speed transport for video production. That wasn’t available when the initial Pac-12 Networks studio was designed in 2012.”
ASG’s commitment to the mission of the new Pac-12 Network studio was behind Pac-12 Networks’ choice of Systems Integrator, along with a level of trust. With such a large-scale project, they needed a firm that would be collaborative but also speak up when needed.
Another key factor was the ability to establish a level of trust between the two organizations, while operating within a cost transparent structure and an accelerated timeline. ASG stayed within budget throughout the entire project.
Van Hoy concluded that the cooperation on all fronts – engineering, operations, and business – from Pac-12 Networks and the tech vendors was remarkable: “It was truly a partnership. We’re so proud to have been involved in what is as forward-looking a facility as exists anywhere in the United States.”
