Mistral AI and NVIDIA have unveiled the Mistral NeMo 12B, a state-of-the-art enterprise AI model that promises to revolutionize various applications, including chatbots, multilingual tasks, coding, and summarization. This cutting-edge language model is designed for easy customization and deployment, offering high performance and reliability for diverse enterprise needs.
The collaboration between Mistral AI and NVIDIA combines Mistral AI’s expertise in training data with NVIDIA’s optimized hardware and software ecosystem. Guillaume Lample, cofounder and chief scientist of Mistral AI, expressed enthusiasm about the partnership, highlighting the unprecedented accuracy, flexibility, high efficiency, and enterprise-grade support and security provided by NVIDIA AI Enterprise deployment.
Mistral NeMo was trained on the NVIDIA DGX Cloud AI platform, which provides dedicated, scalable access to the latest NVIDIA architecture. The model also leverages NVIDIA TensorRT-LLM for accelerated inference performance on large language models and the NVIDIA NeMo development platform for building custom generative AI models. This collaboration underscores NVIDIA’s commitment to supporting the model-builder ecosystem.
Excelling in multi-turn conversations, math, common sense reasoning, world knowledge, and coding, this enterprise-grade AI model delivers precise and reliable performance across diverse tasks. With a 128K context length, Mistral NeMo processes extensive and complex information more coherently and accurately, ensuring contextually relevant outputs.
Released under the Apache 2.0 license, which fosters innovation and supports the broader AI community, Mistral NeMo is a 12-billion-parameter model. The model uses the FP8 data format for inference, reducing memory size and speeding deployment without any degradation in accuracy. This means the model learns tasks better and handles diverse scenarios more effectively, making it ideal for enterprise use cases.
Mistral NeMo comes packaged as an NVIDIA NIM inference microservice, offering performance-optimized inference with NVIDIA TensorRT-LLM engines. This containerized format allows for easy deployment anywhere, providing enhanced flexibility for various applications. As a result, models can be deployed in minutes rather than days.
NIM features enterprise-grade software that’s part of NVIDIA AI Enterprise, with dedicated feature branches, rigorous validation processes, and enterprise-grade security and support. It includes comprehensive support, direct access to an NVIDIA AI expert, and defined service-level agreements, delivering reliable and consistent performance. The open model license allows enterprises to integrate Mistral NeMo into commercial applications seamlessly.
Designed to fit on the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090, or NVIDIA RTX 4500 GPU, the Mistral NeMo NIM offers high efficiency, low compute cost, and enhanced security and privacy. The combined expertise of Mistral AI and NVIDIA engineers has optimized training and inference for Mistral NeMo.
Trained with Mistral AI’s expertise, especially in multilinguality, code, and multi-turn content, the model benefits from accelerated training on NVIDIA’s full stack. It’s designed for optimal performance, utilizing efficient model parallelism techniques, scalability, and mixed precision with Megatron-LM.
The model was trained using Megatron-LM, part of NVIDIA NeMo, with 3,072 H100 80GB Tensor Core GPUs on DGX Cloud, composed of NVIDIA AI architecture, including accelerated computing, network fabric, and software to increase training efficiency. With the flexibility to run anywhere — cloud, data center, or RTX workstation — Mistral NeMo is ready to revolutionize AI applications across various platforms.
Experience Mistral NeMo as an NVIDIA NIM today via ai.nvidia.com, with a downloadable NIM coming soon.
Your First 10 AI Skills
10 practical AI skills, copy-paste prompts and a 7-day plan to start using AI with confidence.
Download the guide →