OpenAI's Open Models Optimized for NVIDIA GPUs: Boosting AI Applications and Democratizing Access

NVIDIA has optimized OpenAI's new open-source gpt-oss models for their GeForce RTX and RTX PRO GPUs, enabling fast inference from the cloud to the PC. The models support agentic AI applications like web search and in-depth research. Developers can use popular tools like Ollama and llama.cpp on NVIDIA RTX AI PCs and workstations, achieving performance of up to 256 tokens per second on the GeForce RTX 5090 GPU.

NVIDIA has optimized OpenAI's new open-source gpt-oss models for their GeForce RTX and RTX PRO GPUs, enabling fast inference from the cloud to the PC. These models, gpt-oss-20b and gpt-oss-120b, support agentic AI applications such as web search and in-depth research. Developers can leverage popular tools like Ollama and llama.cpp on NVIDIA RTX AI PCs and workstations, achieving performance of up to 256 tokens per second on the GeForce RTX 5090 GPU.

The collaboration between NVIDIA and OpenAI underscores the companies' commitment to advancing AI technology and making it accessible to a broader audience. The gpt-oss models were trained on NVIDIA's H100 GPUs and optimized for the NVIDIA CUDA platform, which is the world's most widely available computing infrastructure.

Developers can now use these models on NVIDIA RTX AI PCs and workstations through various applications and frameworks, including Ollama, llama.cpp, and Microsoft AI Foundry Local. Ollama, in particular, offers an easy and optimized way to test the models with out-of-the-box support and customizable context lengths.

The release of these open-source models marks a significant step forward in AI innovation, allowing developers to build on the state-of-the-art open-source foundation provided by OpenAI and NVIDIA. This collaboration highlights NVIDIA's leadership in AI from training to inference and from cloud to AI PC.

References:
[1] https://blogs.nvidia.com/blog/rtx-ai-garage-openai-oss/
[2] https://blogs.nvidia.com/blog/openai-gpt-oss/