Fernando Rejon Barrera
CTO
Zeta Alpha
Presentation Title:
Co-speaking with Jakub Zevrel
Customizing Deep Research:
Elo Evals and Optimization of Advanced AI Agents
Presentation Summary:
How can advanced agentic AI systems like Deep Research be customized on private enterprise data with just a few labeled examples? How can we evaluate and optimize these modular, non-differentiable systems to build high performance reasoning systems for complex tasks?
In this talk we will outline the architecture of this class of agents, show how LLM-as-a-judge can be used for Elo based evaluation in the absence of gold standard benchmarking data sets, and explore how sparse evaluation signals can be used for end-to-end optimization of AI agents to improve performance.
About | Fernando Rejon Barrera
Fernando Rejon Barrera is the Co-Founder and CTO of Zeta Alpha. Zeta Alpha is an Amsterdam-based company specializing in next-generation enterprise AI Discovery Engines for High-Tech and R&D-centric Enterprises.
With a PhD in theoretical physics and a hacker (builder) past, Fernando brings a unique blend of theoretical knowledge and hands-on expertise to his role. He is a generalist with a passion for building innovative solutions and fostering teamwork.
His interests lie in Large Language Models, Embeddings models, Enterprise Search, and Generative AI.