Nvidia's eyeing up ways to beef up its future AI chips with GPUs on top of GPUs and the power of light

pcgamer.com:

I'm not usually one for extravagant tech and hardware predictions—things can change so much and so unpredictably that it's difficult to put too much weight behind such premonitions. But when it comes to AI tech, things move so quickly that what seems distant might not actually be too far away. Combine this with Nvidia being the one outlining the futuristic vision, and I take it a little more seriously.

Nvidia's latest prediction, as outlined at the IEDM 2024 conference according to Dr. Ian Cutress (via TechPowerUp), is AI accelerators that are 3D-stacked and that use—at least in part—silicon photonics for data transmission. This is, as Cutress puts it, Nvidia's vision of «the future of AI compute».

The image provided in the post shows an AI accelerator (ie, a datacentre GPU) that's split vertically into a substrate, integrated silicon photonics, GPU tiers, 3D stacked DRAM, and cold plate.

The two big innovations in this picture, insofar as they could be applied to AI accelerators, are silicon photonics and the vertical stacking for logic. The former uses photons (light) to transmit data to and from optical components, which is faster and uses less power for more bandwidth than traditional electrical data transmission.

Judging by the diagram, it looks like this light-based transmission technology would be used horizontally to connect to other accelerators.

Here's @NVIDIA's vision of the future of AI compute.Silicon photonics interposerSiPh intrachip and interchip12 SiPh connects, 3 per GPU tile4 GPU tiles per tierGPU 'tiers' (GPU on GPU?!?)3D Stacked DRAM, 6 per tile, fine-grainedFrom #iedm24. My guess, 2028/2029/2030… pic.twitter.com/5IsDkYSWT2December 8, 2024

However, TechPowerUp says these accelerators feature «12 SiPh [silicon photonics] connections for intrachip and interchip connections, with three connections per GPU tile across four GPU tiles per tier». And «intra-chip connection» would seem to imply connection between each of these tiles within each tier,