Leveraging LLM as a Judge: Building Shippy for Maritime Domain Awareness

Protecting the world’s oceans requires accurate and reliable information. In high-stakes operational domains like maritime, incorrect answers can lead to significant resource waste and potential harm to personnel. The Skylight team at Ai2 aimed to build an AI agent that could provide real-time maritime domain awareness while ensuring reliability and trustworthiness. Shippy, their AI for this purpose, was designed with a focus on building a system that could be trusted to deliver correct answers within its limits.