AI Evaluations Team Lead

AI Evaluations Team Lead

Location: Gurugram, India

Seniority: Senior (7+ years engineering experience) 

Purpose: Lead the team responsible for building FNZ's AI evaluations framework across both technical and process dimensions, and drive implementation of that framework across AI solutions to ensure rigorous safety, performance, and compliance standards before production deployment.

Key Responsibilities: 

• Lead the team that defines, builds, and evolves FNZ's AI evaluations framework across both technical components and operating processes, aligned to FNZ's six-pillar framework (Task Performance, Safety & Compliance, Efficiency, Groundedness & Reasoning, Robustness, Suitability)

• Establish evaluation standards, methodologies, tooling, and governance processes, and lead implementation of the framework across AI solutions by embedding it into FNZ's SDLC as mandatory release gates

• Represent evaluations function in AI Governance Committee, providing risk assessments and release recommendations

• Build, mentor, and lead a team of evaluation specialists responsible for developing the framework and partnering with AI solution teams to implement it consistently across the estate

• Design and execute complex evaluations for high-risk AI agents; lead red teaming exercises for critical deployments

• Communicate evaluation findings to technical and non-technical stakeholders; influence product roadmaps 

Skills and Experience 

• 7+ years of engineering experience, with 3+ years in AI/ML or security testing and 2+ years in evaluation/red teaming

• Deep understanding of LLM-based agents, RAG architectures, and agentic AI systems (not model training)

• Strong programming background with hands-on experience building evaluation tooling, harnesses, or automated assessment workflows for AI agents and solutions

• Proven ability to design evaluation methodologies and frameworks for probabilistic AI systems, covering both technical measures and operational processes

• Experience embedding evaluation, assurance, or control frameworks into software delivery lifecycles and release governance, ideally within regulated environments

• Leadership experience building or managing cross-functional teams and driving adoption of standards across multiple AI products or solution teams

• Ability to translate safety, risk, and compliance requirements into practical evaluation criteria, controls, and release recommendations for production AI solutions