Multilingual AI Quality Specialist

• Join the Multilingual AI Data Quality space within Spotify's Global Language Quality Program, bringing together multilingual language quality evaluation and AI data quality. • Help ensure Spotify's AI-powered experiences are accurate, culturally relevant, and trustworthy across languages and markets. • Develop evaluation methodologies, high-quality datasets, and quality signals that help improve AI experiences and inform product decisions at scale. • Be based in London or Stockholm with flexibility to work from home.

• Have experience in multilingual quality evaluation, localisation, data curation, annotation, AI evaluation, or related fields, including text-to-text and text-to-speech experiences. • Understand language quality, cultural relevance, content quality, and user experience across multiple languages and markets. • Have experience designing or conducting structured evaluations using quality rubrics, audits, annotation projects, or review methodologies. • Be comfortable using qualitative and quantitative data to identify trends, measure quality, and make recommendations. • Be familiar with large language models, generative AI evaluation, human-in-the-loop workflows, or LLM-as-a-judge methodologies. • Enjoy working through ambiguity and turning complex quality challenges into practical evaluation strategies. • Communicate effectively and thrive in highly cross-functional environments, collaborating with technical and non-technical partners alike.

• Have experience with recommendation systems, personalisation, search, or ranking. • Have experience with machine translation or generative AI. • Have experience with dataset creation, annotation operations, or evaluator calibration. • Have experience with prompt testing or model evaluation. • Have experience with SQL, Python, dashboards, or annotation platforms.

• Define quality frameworks, evaluation rubrics, thresholds, and methodologies for multilingual AI experiences. • Design and execute structured evaluations for AI-generated, AI-translated, AI-curated, and recommendation-driven experiences. • Lead multilingual dataset curation, annotation, enrichment, and ground-truth creation to support AI model development and evaluation. • Analyse evaluation results, identify quality gaps, and provide actionable recommendations to improve multilingual AI quality. • Support LLM-as-a-judge workflows, evaluator calibration, and human-AI agreement studies. • Partner closely with Product, Engineering, Data Science, Research, Localisation, vendors, and market experts to improve AI quality signals and inform launch decisions. • Document best practices and help define quality standards across languages, markets, and AI use cases. • Contribute to building scalable evaluation capabilities that support the next generation of AI-powered experiences across Spotify.

• Work from London or Stockholm with flexibility to work from home. • Receive extensive learning opportunities through Spotify's dedicated GreenHouse team. • Receive flexible share incentives letting you choose how you share in Spotify's success. • Receive six months of global parental leave for all new parents. • Access All The Feels, the employee assistance programme and self-care hub. • Swap public holidays flexibly according to your values and beliefs.

Spotify is the world's most popular audio streaming subscription service with more than 700 million monthly active users. The Platform team creates the technology that enables Spotify to learn quickly and scale easily, building the infrastructure, tooling, frameworks, and capabilities needed to welcome a billion customers.