NVIDIA Brings Up To 5x AI Acceleration To Windows 11 PCs Running RTX 40 & RTX 30 GPUs

wccftech.com:

NVIDIA is bringing a huge acceleration to AI Workloads to millions of Windows 11 PCs powered by its latest RTX GPUs.

Following up on its previous announcement, NVIDIA has now revealed that TensorRT-LLM is being added to Windows 11 and will be enabled for more than 100 million RTX users when it launches in the latest driver suite on the 21st of November. The announcement was made during Microsoft's Ignite, a key event discussing the future of AI and how it will transform the Windows ecosystem as we move forward.

Today, NVIDIA confirmed that TensorRT-LLM AI acceleration will be available for all RTX Desktops & laptops with more than 8 GB of VRAM. In addition to TensorRT-LLM, NVIDIA and Microsoft are also bringing DirectML enhancements to boost popular AI models such as Stable Diffusion and Llama 2.

Having an NVIDIA RTX GPU that supports TensorRT-LLM means that you will have all your data and projects available locally rather than saving them in the cloud. This would save time & deliver more precise results. RAG or Retrieval Augamanted Generation is one of the techniques used in making AI results faster by using a localized library that can be filled with the dataset you want the LLM to go through & then leverage the language understating capabilities of that LLM to provide you with accurate results.

NVIDIA states a 5x performance boost with TensorRT-LLM v0.6.0 which will be available later this month. Furthermore, it will also enable support for additional LLMs such as Mistral 7B & Nemotron 3 8B.

For those who want to try out the latest release of TensorRT-LLM, it will be available for installation at the official Github link here & you can also grab the latest optimized models from NVIDIA's NGC resource.

Another key update is