Broadcom: Hyperscalers Bypassing Telecom Operators in Scale Cross Project

Telecom operators are generally eager to establish fast fiber optic connections between data centers for large-scale enterprises and Neocloud, and several protocols have been implemented. Operators hope to integrate fiber optic, optical line systems, switches, and various services to connect large-scale GPU clusters in dispersed data centers. And Broadcom, the manufacturer of most high-capacity switch chips, is seeing a steady increase in demand for such connections.

Hasan Siraj, Vice President of Product at Broadcom, said, “Demand has been on the rise in the past two years as cluster sizes continue to grow. Some ultra large enterprises are building and controlling these connections themselves. But if they do need intermediate service providers, they will definitely use them.”

The connection between AI data centers is commonly referred to as “Scale cross”, distinguishing it from “Scale up” (internal connection of AI server racks) and “Scale out” (connection between racks). Siraj stated that Broadcom provides solutions covering all three types of application scenarios.

For the Scale cross architecture, hardware level security is a mandatory requirement. Siraj pointed out that Broadcom’s Jericho switching chip has deep buffering and line speed encryption capabilities, and uses telemetry technology to expose anomalies or denial of service attacks.

Network is Computer

Siraj is responsible for Broadcom’s routing, switching, high-performance NIC, and open network software product portfolio. He stated that in AI infrastructure, “the network is the computer. When you are training these models or conducting distributed inference, you cannot adapt the models to a single XPU, “he told Fierce.” You need to connect tens, hundreds, hundreds of thousands, or even 500 million units based on the task being performed. Therefore, you need to connect all these components, and that is the network.”

Even with rising computational costs, Siraj believes that it is unlikely for large-scale enterprises to significantly reduce their network budgets. Service providers and large-scale enterprises hope to complete network deployment three to six months before the computing devices are in place, “he said.” They will first build the network. When you sit in the rack, the connection will be achieved immediately after the computing devices arrive.”

Siraj stated that a 10 megawatt data center can accommodate approximately 7000 XPUs/GPUs/TPUs. If the use case requirements are larger, it is necessary to connect the data centers together. “These distances may range from tens of kilometers to about 100 kilometers.

EJL Wireless Research analyst Earl Lum agrees that data center interconnectivity is necessary for training AI models, but he is unsure whether distributed inference will become the main driving factor for Scale cross connectivity. You need Scale cross to aggregate data centers together to train these models and utilize all computing resources, so you need to mesh them all together, “he said.” But if you no longer need this type of training, you may no longer need those fiber optic interconnections. Lum believes that the portion of AI workloads dedicated to inference is on the rise relative to training.

No matter how scale cross connections develop, AI is expected to drive a surge in demand for internal networks in data centers. Siraj stated that due to the requirement for close communication between switches and network interface cards in AI architecture, Broadcom has developed an AI NIC.

The timing is just right

Broadcom’s move comes at a perfect time, as GPUs running AI workloads use NICs to communicate directly with each other. The demand for such network cards has exploded, and the supply of AI network interface cards is becoming increasingly tight. The dominant Nvidia ConnectX network card in the market is difficult to obtain except for the largest buyer. Siraj stated that Broadcom has a relatively abundant supply of Thor 2 and NetXtreme solutions. “For us, this is a new market that we are entering, and the overall demand is indeed extremely high, but for the customers we are working with, we are able to meet their needs.”

Unlike Nvidia network cards that use AI specific InfiniBand connectivity architecture, Broadcom’s network cards use Ethernet. Siraj said, “I think there is almost a consensus in the industry now that Ethernet should become the de facto standard for most use cases.” This echoes previous predictions from other non Nvidia vendors.