Senior Data Scientist

Job Summary

The Senior Data Scientist is responsible for leading experimentation, model evaluation, and feature strategy for the Topic Ranking GenAI Personalization initiative. This role focuses on designing and validating personalization approaches that combine behavioral ML signals with GenAI-derived user profiles, using rigorous ranking evaluation methods and business-impact analysis to guide model selection.

Key Responsibilities

• Design and execute the proof of concept for comparing behavioral feature engineering and GenAI-enriched user profiling approaches.
• Develop prompt strategies for generating structured user profiles from ordering and behavioral history.
• Define the experimentation framework used to assess ranking quality and personalization lift.
• Conduct offline evaluation using ranking metrics such as NDCG, MRR, Precision, and Recall.
• Perform ablation studies to isolate the impact of specific feature sets and enrichment methods.
• Lead hyperparameter optimization and support feature selection decisions.
• Apply bias-correction techniques, including inverse propensity scoring, to improve training and evaluation quality.
• Analyze performance by user cohort, segment, or behavioral profile to identify where personalization creates the most value.
• Partner with engineering and product stakeholders to map model outcomes to business KPIs and experimentation readiness.

Required Skills

• Strong background in machine learning, recommendation systems, ranking models, or personalization.
• Expertise in experimentation design, offline evaluation, and statistical analysis.
• Experience with ranking metrics such as NDCG, MRR, Precision, and Recall.
• Proficiency in Python and/or SQL for feature engineering, analysis, and evaluation workflows.
• Familiarity with LLM prompt design or GenAI-driven feature generation approaches.
• Experience with hyperparameter tuning methodologies such as Bayesian optimization or grid search.
• Ability to synthesize technical findings into clear recommendations for product and business stakeholders.

Preferred Qualifications

• Experience in consumer-facing recommendation or search/ranking systems.
• Familiarity with multi-objective optimization involving CTR, CVR, diversity, or engagement metrics.
• Experience working with behavioral event data, clickstream data, or marketplace/user-intent data.
• Exposure to shadow testing, guardrail metrics, and A/B test preparation.
• Prior experience with AWS-native ML environments or SageMaker-based experimentation.