Amazon Nova LLM-as-a-Judge on SageMaker AI: A New Standard for Evaluating Generative AI Models

Amazon Nova LLM-as-a-Judge on Amazon SageMaker AI provides a comprehensive approach to model evaluation for generative AI models. It delivers robust, unbiased assessments of generative AI outputs across model families and has been rigorously validated to remain impartial. The capability provides pairwise comparisons between model iterations, enabling data-driven decisions about model improvements with confidence. Nova LLM-as-a-Judge was built through a multistep training process comprising supervised training and reinforcement learning stages that used public datasets annotated with human preferences.