Build Evals That Actually Matter - Nick Ung, Lyft

Summary

This talk focuses on building effective AI agents for customer support, specifically at Lyft. The presenters discuss the importance of rigorous evaluation, covering the end-to-end pipeline from development to production and introducing concepts like offline and online evaluations. The key takeaway is the necessity of robust evaluation methods to ensure AI agent performance before deploying them to live users.