GenAI Engineering Leader

Hire and build GenAI engineering teams

• Define GenAI-specific hiring criteria and design technical interviews for LLM and agent engineering roles
• Build skill assessment frameworks and team composition strategies balancing generalist and specialist profiles
• Write job descriptions, design interview rubrics, and evaluate candidates against GenAI competency matrices

Define engineering processes

for GenAI development — eval-driven workflows

• Design GenAI-specific sprint planning with eval-driven development as the core feedback loop
• Define evaluation metrics before writing code and measure GenAI team velocity with non-deterministic outputs
• Build team workflows integrating Langfuse for evaluation tracking and Grafana for velocity metrics

Manage quality and team performance

for GenAI outputs

• Define GenAI quality metrics and SLA management frameworks for LLM system reliability
• Build team performance dashboards using Grafana with latency, quality, and throughput indicators
• Construct performance dashboards and define quality standards for GenAI engineering deliverables

Understand the technical stack

deeply enough to unblock teams

• Learn LLM fundamentals, LangGraph agent engineering patterns, and LiteLLM gateway operations
• Monitor production systems with Langfuse and Prometheus to review PRs and debug incidents
• Gain sufficient depth to make architecture calls, review designs, and unblock teams on technical decisions

Operate and budget for GenAI infrastructure

— FinOps and capacity

• Build LLM cost attribution dashboards with capacity planning and budget forecasting models
• Manage vendor relationships and optimize spend allocation across multiple LLM providers
• Construct FinOps dashboards, set team-level token budgets, and produce monthly cost reports for leadership

Design organization structure

for GenAI engineering teams

• Apply GenAI team topology patterns including on-call rotation design and knowledge sharing practices
• Evaluate embed-vs-centralize tradeoffs for GenAI engineering functions across the organization
• Design org structures for different company sizes with clear ownership boundaries and escalation paths

Drive technical strategy

— evaluate new tools and plan migrations

• Apply technology evaluation frameworks with structured criteria for GenAI tool and platform selection
• Build migration planning methodology and strategic roadmaps for technology transitions
• Evaluate new tools against defined criteria, build migration plans, and present strategy to leadership

Ensure responsible AI practices

across your team

• Design governance policies and safety review processes for GenAI system development and deployment
• Build compliance workflows and team-level responsible AI standards with enforcement mechanisms
• Create governance policies and integrate safety review checkpoints into the development lifecycle