Managing AI Teams

Managing AI teams requires adapting standard engineering management practices in several key ways. The work is inherently more experimental: a significant percentage of ML experiments will produce negative results, and that is normal and expected. Feedback loops are longer: model training, evaluation, and iteration cycles take days or weeks rather than hours. Skill profiles are more diverse: AI teams need researchers, ML engineers, data engineers, and MLOps specialists who have different working styles, career expectations, and evaluation criteria. Understanding these differences is prerequisite to managing an AI team effectively.

The first hiring decision is the team's skill mix. Most teams need more ML engineers (who bridge research and production) than pure researchers or pure data scientists. The ideal early team has ML engineers who can train models AND deploy them to production, supplemented by data engineers who can build reliable data pipelines. Hire pure researchers only when your problem requires novel model architectures or approaches, not when existing models need to be adapted and deployed.

Traditional OKRs map poorly to AI work because outcomes are uncertain. A team can do excellent work on a well-designed experiment and still produce a negative result. Adapt goal-setting by separating delivery goals from learning goals. Delivery goals are standard: ship feature X with Y quality by date Z. Learning goals acknowledge uncertainty: run experiment X to test hypothesis Y, report results by date Z regardless of outcome. Both types of goals are first-class contributions.

Evaluating AI practitioners requires adjusting your calibration framework. An ML engineer who runs three well-designed experiments that all produce negative results has contributed valuable knowledge to the organization. If you only reward positive experimental outcomes, you incentivize skipping evaluation and shipping models that are not ready. Evaluate based on: the rigor of the experimental approach, the quality of the analysis, the clarity of the documentation, and the contribution to organizational learning. Positive business outcomes are a team metric, not an individual performance metric for experimental work.

AI sprint planning requires accommodations that standard product sprints do not. Explicitly allocate capacity for experimentation (20-30% of sprint capacity). Accept that some work items will span multiple sprints (training runs, large-scale evaluations). Create a process for handling 'blocked on data' situations, which are far more common in AI teams than in product engineering teams. Use the AI Sprint Planning Agenda template to structure sprint planning sessions that balance feature delivery with exploration.

AI work creates stakeholder management challenges that product engineering does not. Timelines are less predictable, outcomes are uncertain, and progress is harder to demonstrate. Address this by: setting expectations about experimental uncertainty upfront, providing regular progress updates that include negative results (framed as valuable learning), using demos strategically (only demo when the system is representative of final quality, not before), and maintaining a backlog of 'safe wins' that can be delivered when experimental work stalls.

AI practitioners need career paths that value both research and engineering contributions. An ML engineer who builds production-grade infrastructure should be able to advance as fast as one who publishes research. Define career levels that account for both tracks: the engineering track emphasizes production systems, reliability, and team impact; the research track emphasizes novel approaches, evaluation methodology, and knowledge contribution. Allow movement between tracks as interests evolve.