MSDS Seminar Series in Data Science: Maxine Qian, Data Scientist, Experimentation, Pinterest
Friday, October 1 12:30 – 2 p.m.
101 Howard St. Rooms 154-156
View via Zoom
Maxine Qian is a Senior Data Scientist at Pinterest, leading the offline experimentation efforts in the Experimentation and Metrics Science team. She also worked on online experimentation quality and ads measurement solutions. Her work focuses on developing new methodologies and improving velocity and quality in experimentation. She received the M.S. degree in Data Science from USF in 2017, and the B.S. degree in Economics from Wuhan University in 2012.
Running online experiments can be slow and expensive. Picking a few worth running experiments from a large set of candidates relies heavily on subjective choices and intuition. In some situations, we sequentially iterate through numerous experimental candidates and wait for weeks to get results back. In others, a carefully tuned model with AUC improvement disappoints with a negative experiment result.
We developed an offline experimentation framework, which allows us to select optimal candidates in a matter of hours instead of days required for online experiments. It can forecast the output of key metrics offline through a replay approach on past data. This framework enables us to quickly iterate product design and provide better user experience. The talk will cover the methodology, empirical results and use cases of the framework.