"Astera contends that with a big enough switch, PCIe is a viable alternative to interconnects like NVLink, in the scale-up fabrics used to make dozens or more GPUs behave more like a single large one without needing to redesign their accelerators."
"By moving collective communications to the switch, the GPUs spend less time waiting for the network to catch up and more time churning out tokens."
"Astera has gone so far as to develop a multicast operation optimized for MoE inference that it calls Hypercast."
Astera Labs launched Scorpio X, a PCIe switch designed to replace Nvidia's NVSwitch for AI systems. It features 320 lanes of PCIe 6.0 connectivity and 5.12 TB/s bidirectional bandwidth. This switch allows for efficient connections among GPUs, NICs, and storage, making PCIe a viable alternative to NVLink. Scorpio X includes in-network compute capabilities to accelerate collective communications, crucial for generative AI inference. Astera developed Hypercast, a multicast operation optimized for mixture-of-experts models, improving efficiency in token generation by reducing network wait times.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]