A single AI agent conversation can look perfect and still be broken, leaders from LangChain, Conviva and CoreWeave said at VB Transform 2026

Enterprise AI agent evaluation is shifting from scoring individual interactions to cohort-based analysis comparing user populations against baselines, revealing failures that single-trace scoring misses and driving a move toward smaller, cheaper judge models rather than relying solely on large LLMs. IT leaders must recognize that evaluation criteria now function as living product specifications (comparable to PRDs) requiring continuous iteration post-launch rather than exhaustive pre-deployment testing, and that automated judging cannot fully replace human oversight in regulated industries. This fundamentally changes how organizations should architect AI observability and governance—prioritizing broad, always-on monitoring to identify failure patterns in production before building targeted offline evaluation sets.